Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Efficient Decoding of Compositional Structure in Holistic Representations
RISE Research Institutes of Sweden, Digitala system, Datavetenskap. University of California, USA.ORCID-id: 0000-0002-6032-6155
University of California, USA.
University of California, USA.
University of California, USA.
Vise andre og tillknytning
2023 (engelsk)Inngår i: Neural Computation, ISSN 0899-7667, E-ISSN 1530-888X, Vol. 35, nr 7, s. 1159-1186Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

We investigate the task of retrieving information from compositional distributed representations formed by hyperdimensional computing/vector symbolic architectures and present novel techniques that achieve new information rate bounds. First, we provide an overview of the decoding techniques that can be used to approach the retrieval task. The techniques are categorized into four groups. We then evaluate the considered techniques in several settings that involve, for example, inclusion of external noise and storage elements with reduced precision. In particular, we find that the decoding techniques from the sparse coding and compressed sensing literature (rarely used for hyperdimensional computing/vector symbolic architectures) are also well suited for decoding information from the compositional distributed representations. Combining these decoding techniques with interference cancellation ideas from communications improves previously reported bounds (Hersche et al., 2021) of the information rate of the distributed representations from 1.20 to 1.40 bits per dimension for smaller codebooks and from 0.60 to 1.26 bits per dimension for larger codebooks. 

sted, utgiver, år, opplag, sider
MIT Press Journals , 2023. Vol. 35, nr 7, s. 1159-1186
Emneord [en]
Architecture, Computer architecture, Codebooks, Compositional structure, Decoding techniques, Distributed representation, External noise, Four-group, Information rates, Novel techniques, Reduced precision, Storage elements, article, information retrieval, noise, Decoding
HSV kategori
Identifikatorer
URN: urn:nbn:se:ri:diva-65998DOI: 10.1162/neco_a_01590Scopus ID: 2-s2.0-85163619213OAI: oai:DiVA.org:ri-65998DiVA, id: diva2:1790363
Merknad

We thank Spencer Kent for sharing with us the implementation of the FISTAalgorithm. The work of F.T.S., B.A.O., C.B., and D.K. was supported in partby Intel’s THWAI program. The work of B.A.O. and D.K. was also supported in part by AFOSR FA9550-19-1-0241. D.K. has received funding fromthe European Union’s Horizon 2020 research and innovation program. under Marie Sklodowska-Curie grant agreement 839179. The work of C.J.K.was supported by the U.S. Department of Defense through the NationalDefense Science and Engineering Graduate Fellowship Program. F.T.S. wassupported by Intel and NIH R01-EB026955.

Tilgjengelig fra: 2023-08-22 Laget: 2023-08-22 Sist oppdatert: 2023-12-12bibliografisk kontrollert

Open Access i DiVA

Fulltekst mangler i DiVA

Andre lenker

Forlagets fulltekstScopus

Person

Kleyko, Denis

Søk i DiVA

Av forfatter/redaktør
Kleyko, Denis
Av organisasjonen
I samme tidsskrift
Neural Computation

Søk utenfor DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric

doi
urn-nbn
Totalt: 3 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
v. 2.41.0