Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning

Semih Yagli, Alex Dytso, H. Vincent Poor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Scopus citations

Abstract

Machine learning algorithms operating on mobile networks can be characterized into three different categories. First is the classical situation in which the end-user devices send their data to a central server where this data is used to train a model. Second is the distributed setting in which each device trains its own model and send its model parameters to a central server where these model parameters are aggregated to create one final model. Third is the federated learning setting in which, at any given time t, a certain number of active end users train with their own local data along with feedback provided by the central server and then send their newly estimated model parameters to the central server. The server, then, aggregates these new parameters, updates its own model, and feeds the updated parameters back to all the end users, continuing this process until it converges.The main objective of this work is to provide an information-Theoretic framework for all of the aforementioned learning paradigms. Moreover, using the provided framework, we develop upper and lower bounds on the generalization error together with bounds on the privacy leakage in the classical, distributed and federated learning settings.

Original languageEnglish (US)
Title of host publication2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728154787
DOIs
StatePublished - May 2020
Event21st IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2020 - Atlanta, United States
Duration: May 26 2020May 29 2020

Publication series

NameIEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC
Volume2020-May

Conference

Conference21st IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2020
Country/TerritoryUnited States
CityAtlanta
Period5/26/205/29/20

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Computer Science Applications
  • Information Systems

Fingerprint

Dive into the research topics of 'Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning'. Together they form a unique fingerprint.

Cite this