Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Efficient Decoding of Compositional Structure in Holistic Representations
RISE Research Institutes of Sweden, Digital Systems, Data Science. University of California, USA.ORCID iD: 0000-0002-6032-6155
University of California, USA.
University of California, USA.
University of California, USA.
Show others and affiliations
2023 (English)In: Neural Computation, ISSN 0899-7667, E-ISSN 1530-888X, Vol. 35, no 7, p. 1159-1186Article in journal (Refereed) Published
Abstract [en]

We investigate the task of retrieving information from compositional distributed representations formed by hyperdimensional computing/vector symbolic architectures and present novel techniques that achieve new information rate bounds. First, we provide an overview of the decoding techniques that can be used to approach the retrieval task. The techniques are categorized into four groups. We then evaluate the considered techniques in several settings that involve, for example, inclusion of external noise and storage elements with reduced precision. In particular, we find that the decoding techniques from the sparse coding and compressed sensing literature (rarely used for hyperdimensional computing/vector symbolic architectures) are also well suited for decoding information from the compositional distributed representations. Combining these decoding techniques with interference cancellation ideas from communications improves previously reported bounds (Hersche et al., 2021) of the information rate of the distributed representations from 1.20 to 1.40 bits per dimension for smaller codebooks and from 0.60 to 1.26 bits per dimension for larger codebooks. 

Place, publisher, year, edition, pages
MIT Press Journals , 2023. Vol. 35, no 7, p. 1159-1186
Keywords [en]
Architecture, Computer architecture, Codebooks, Compositional structure, Decoding techniques, Distributed representation, External noise, Four-group, Information rates, Novel techniques, Reduced precision, Storage elements, article, information retrieval, noise, Decoding
National Category
Telecommunications
Identifiers
URN: urn:nbn:se:ri:diva-65998DOI: 10.1162/neco_a_01590Scopus ID: 2-s2.0-85163619213OAI: oai:DiVA.org:ri-65998DiVA, id: diva2:1790363
Note

We thank Spencer Kent for sharing with us the implementation of the FISTAalgorithm. The work of F.T.S., B.A.O., C.B., and D.K. was supported in partby Intel’s THWAI program. The work of B.A.O. and D.K. was also supported in part by AFOSR FA9550-19-1-0241. D.K. has received funding fromthe European Union’s Horizon 2020 research and innovation program. under Marie Sklodowska-Curie grant agreement 839179. The work of C.J.K.was supported by the U.S. Department of Defense through the NationalDefense Science and Engineering Graduate Fellowship Program. F.T.S. wassupported by Intel and NIH R01-EB026955.

Available from: 2023-08-22 Created: 2023-08-22 Last updated: 2023-12-12Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Kleyko, Denis

Search in DiVA

By author/editor
Kleyko, Denis
By organisation
Data Science
In the same journal
Neural Computation
Telecommunications

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 4 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf