Computing on Functions Using Randomized Vector Representations (in brief) Show others and affiliations
2022 (English) In: ACM International Conference Proceeding Series, Association for Computing Machinery , 2022, p. 115-122Conference paper, Published paper (Refereed)
Abstract [en]
Vector space models for symbolic processing that encode symbols by random vectors have been proposed in cognitive science and connectionist communities under the names Vector Symbolic Architecture (VSA), and, synonymously, Hyperdimensional (HD) computing [22, 31, 46]. In this paper, we generalize VSAs to function spaces by mapping continuous-valued data into a vector space such that the inner product between the representations of any two data points approximately represents a similarity kernel. By analogy to VSA, we call this new function encoding and computing framework Vector Function Architecture (VFA). In VFAs, vectors can represent individual data points as well as elements of a function space (a reproducing kernel Hilbert space). The algebraic vector operations, inherited from VSA, correspond to well-defined operations in function space. Furthermore, we study a previously proposed method for encoding continuous data, fractional power encoding (FPE), which uses exponentiation of a random base vector to produce randomized representations of data points and fulfills the kernel properties for inducing a VFA. We show that the distribution from which components of the base vector are sampled determines the shape of the FPE kernel, which in turn induces a VFA for computing with band-limited functions. In particular, VFAs provide an algebraic framework for implementing large-scale kernel machines with random features, extending [51]. Finally, we demonstrate several applications of VFA models to problems in image recognition, density estimation and nonlinear regression. Our analyses and results suggest that VFAs constitute a powerful new framework for representing and manipulating functions in distributed neural systems, with myriad potential applications in artificial intelligence.
Place, publisher, year, edition, pages Association for Computing Machinery , 2022. p. 115-122
Keywords [en]
Holographic Reduced Representations, Hyperdimensional Computing, Kernel Methods, Population Coding, Theoretical Neuroscience, Vector Symbolic Architectures, Architecture, Computer architecture, Encoding (symbols), Image recognition, Signal encoding, Vector spaces, Basis vector, Datapoints, Fractional power, Function spaces, Kernel-methods, Vector functions, Vector symbolic architecture, Vectors
National Category
Biochemistry Molecular Biology
Identifiers URN: urn:nbn:se:ri:diva-59335 DOI: 10.1145/3517343.3522597 Scopus ID: 2-s2.0-85130042092 ISBN: 9781450395595 (electronic) OAI: oai:DiVA.org:ri-59335 DiVA, id: diva2:1674860
Conference 2022 Annual Neuro-Inspired Computational Elements Conference, NICE 2022, 28 March 2022 through 1 April 2022
Note Funding details: National Institutes of Health, NIH, R01-EB026955; Funding details: U.S. Department of Defense, DOD; Funding details: Air Force Office of Scientific Research, AFOSR, FA9550-19-1-0241; Funding details: Intel Corporation; Funding details: H2020 Marie Skłodowska-Curie Actions, MSCA, 839179; Funding details: National Defense Science and Engineering Graduate, NDSEG; Funding text 1: The work of FTS, BAO, and DK was supported in part by Intel's THWAI program. The work of BAO and DK was supported in part by AFOSR FA9550-19-1-0241. The work of DK was supported in part by the European Union's Horizon 2020 Programme under the Marie Sklodowska-Curie Individual Fellowship Grant (839179). The work of CJK was supported by the Department of Defense (DoD) through the National Defense Science & Engineering Graduate (NDSEG) Fellowship Program. FTS was supported by Intel and NIH R01-EB026955.; Funding text 2: The work of FTS, BAO, and DK was supported in part by Intel’s THWAI program. The work of BAO and DK was supported in part by AFOSR FA9550-19-1-0241. The work of DK was supported in part by the European Union’s Horizon 2020 Programme under the Marie Skłodowska-Curie Individual Fellowship Grant (839179). The work of CJK was supported by the Department of Defense (DoD) through the National Defense Science & Engineering Graduate (NDSEG) Fellowship Program. FTS was supported by Intel and NIH R01-EB026955.
2022-06-222022-06-222025-02-20 Bibliographically approved