Adaptive Expert Models for Federated LearningShow others and affiliations
2023 (English)In: Lecture Notes in Computer Science Volume 13448 Pages 1 - 16 2023, Springer Science and Business Media Deutschland GmbH , 2023, p. 1-16Conference paper, Published paper (Refereed)
Abstract [en]
Federated Learning (FL) is a promising framework for distributed learning when data is private and sensitive. However, the state-of-the-art solutions in this framework are not optimal when data is heterogeneous and non-IID. We propose a practical and robust approach to personalization in FL that adjusts to heterogeneous and non-IID data by balancing exploration and exploitation of several global models. To achieve our aim of personalization, we use a Mixture of Experts (MoE) that learns to group clients that are similar to each other, while using the global models more efficiently. We show that our approach achieves an accuracy up to 29.78% better than the state-of-the-art and up to 4.38% better compared to a local model in a pathological non-IID setting, even though we tune our approach in the IID setting. © 2023, The Author(s)
Place, publisher, year, edition, pages
Springer Science and Business Media Deutschland GmbH , 2023. p. 1-16
Keywords [en]
Federated learning, Personalization, Privacy preserving, Artificial intelligence, Learning systems, Distributed learning, Expert modeling, Global models, Heterogeneous data, IID data, Personalizations, Robust approaches, State of the art, Privacy-preserving techniques
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:ri:diva-64398DOI: 10.1007/978-3-031-28996-5_1Scopus ID: 2-s2.0-85152565856ISBN: 9783031289958 (print)OAI: oai:DiVA.org:ri-64398DiVA, id: diva2:1755550
Conference
1st International Workshop on Trustworthy Federated Learning in Conjunction with International Joint Conference on AI, FL-IJCAI 2022. Vienna 23 July 2022 through 23 July 2022
Note
Funding details: Linköpings Universitet, LiU; Funding details: Knut och Alice Wallenbergs Stiftelse; Funding text 1: This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. The computations were enabled by the supercomputing resource Berzelius provided by National Supercomputer Centre at Linköping University and the Knut and Alice Wallenberg foundation. Funding text 2: Acknowledgment. This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
2023-05-082023-05-082024-05-21Bibliographically approved