Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Decentralized and Adaptive K-Means Clustering for Non-IID Data Using HyperLogLog Counters
RISE Research Institutes of Sweden, Digital Systems.
RISE Research Institutes of Sweden, Digital Systems, Data Science.ORCID iD: 0000-0003-4516-7317
Halmstad University, Sweden.
Halmstad University, Sweden.ORCID iD: 0000-0003-3272-4145
Show others and affiliations
2020 (English)In: 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020(Lecture Notes in Computer Science book series (LNCS, volume 12084)), Springer , 2020, p. 343-355Conference paper, Published paper (Refereed)
Abstract [en]

The data shared over the Internet tends to originate from ubiquitous and autonomous sources such as mobile phones, fitness trackers, and IoT devices. Centralized and federated machine learning solutions represent the predominant way of providing smart services for users. However, moving data to central location for analysis causes not only many privacy concerns, but also communication overhead. Therefore, in certain situations machine learning models need to be trained in a collaborative and decentralized manner, similar to the way the data is originally generated without requiring any central authority for data or model aggregation. This paper presents a decentralized and adaptive k-means algorithm that clusters data from multiple sources organized in peer-to-peer networks. Our algorithm allows peers to reach an approximation of the global model without sharing any raw data. Most importantly, we address the challenge of decentralized clustering with skewed non-IID data and asynchronous computations by integrating HyperLogLog counters with k-means algorithm. Furthermore, our clustering algorithm allows nodes to individually determine the number of clusters that fits their local data. Results using synthetic and real-world datasets show that our algorithm outperforms state-of-the-art decentralized k-means algorithms achieving accuracy gain that is up-to 36%. 

Place, publisher, year, edition, pages
Springer , 2020. p. 343-355
Keywords [en]
Approximation algorithms, Data mining, Data Sharing, Machine learning, Peer to peer networks, Central locations, Communication overheads, Machine learning models, Model aggregations, Number of clusters, Privacy concerns, Real-world datasets, State of the art, K-means clustering
National Category
Natural Sciences
Identifiers
URN: urn:nbn:se:ri:diva-45104DOI: 10.1007/978-3-030-47426-3_27Scopus ID: 2-s2.0-85085735657ISBN: 9783030474256 (print)OAI: oai:DiVA.org:ri-45104DiVA, id: diva2:1451090
Conference
24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020
Note

Funding text 1: This research has been conducted within the ?BIDAF: A Big Data Analytics Framework for a Smart Society? (http://bidaf.sics.se/) project funded by the Swedish Knowledge Foundation.

Available from: 2020-07-02 Created: 2020-07-02 Last updated: 2025-09-23Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Girdzijauskas, SarunasPashami, Sepideh

Search in DiVA

By author/editor
Girdzijauskas, SarunasPashami, Sepideh
By organisation
Digital SystemsData Science
Natural Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 46 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf