Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
HyperEmbed: Tradeoffs between Resources and Performance in NLP Tasks with Hyperdimensional Computing Enabled Embedding of n-gram Statistics
Luleå University of Technology, Sweden.
ETH Zürich, Switzerland.
RISE Research Institutes of Sweden, Digital Systems, Data Science.ORCID iD: 0000-0002-6032-6155
Luleå University of Technology, Sweden.
Show others and affiliations
2021 (English)In: Proceedings of the International Joint Conference on Neural Networks, Institute of Electrical and Electronics Engineers Inc. , 2021, Vol. 2021-JulyConference paper, Published paper (Refereed)
Abstract [en]

Recent advances in Deep Learning have led to a significant performance increase on several NLP tasks, however, the models become more and more computationally demanding. Therefore, this paper tackles the domain of computationally efficient algorithms for NLP tasks. In particular, it investigates distributed representations of n -gram statistics of texts. The representations are formed using hyperdimensional computing enabled embedding. These representations then serve as features, which are used as input to standard classifiers. We investigate the applicability of the embedding on one large and three small standard datasets for classification tasks using nine classifiers. The embedding achieved on par F_1 scores while decreasing the time and memory requirements by several times compared to the conventional n -gram statistics, e.g., for one of the classifiers on a small dataset, the memory reduction was 6.18 times; while train and test speed-ups were 4.62 and 3.84 times, respectively. For many classifiers on the large dataset, memory reduction was ca. 100 times and train and test speed-ups were over 100 times. Importantly, the usage of distributed representations formed via hyperdimensional computing allows dissecting strict dependency between the dimensionality of the representation and n-gram size, thus, opening a room for tradeoffs. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2021. Vol. 2021-July
Keywords [en]
Commerce; Deep learning; Embeddings; Large dataset; Natural language processing systems; Statistical tests, Distributed representation; Embeddings; Gram statistic; Hyperdimensional computing; Intent classification; Memory reduction; N-gram statistics; Performance; Test speed; Train’s speed, Classification (of information)
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:ri:diva-68292DOI: 10.1109/IJCNN52387.2021.9534359Scopus ID: 2-s2.0-85108654382OAI: oai:DiVA.org:ri-68292DiVA, id: diva2:1817591
Conference
International Joint Conference on Neural Networks, IJCNN 2021 Virtual, Shenzhen, China. 18 July 2021 through 22 July 2021
Available from: 2023-12-06 Created: 2023-12-06 Last updated: 2023-12-12Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Kleyko, Denis

Search in DiVA

By author/editor
Kleyko, Denis
By organisation
Data Science
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 219 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf