Device sampling for heterogeneous federated learning: Theory, algorithms, and implementation

Su Wang, Mengyuan Lee, Seyyedali Hosseinalipour, Roberto Morabito, Mung Chiang, Christopher G. Brinton

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may contain heterogeneous communication/computation resources, while (ii) there may be significant overlaps in devices' local data distributions. In this work, we develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading. Our optimization aims to select the best combination of sampled nodes and data offloading configuration to maximize FedL training accuracy subject to realistic constraints on the network topology and device capabilities. Theoretical analysis of the D2D offloading subproblem leads to new FedL convergence bounds and an efficient sequential convex optimizer. Using this result, we develop a sampling methodology based on graph convolutional networks (GCNs) which learns the relationship between network attributes, sampled nodes, and resulting offloading that maximizes FedL accuracy. Through evaluation on real-world datasets and network measurements from our IoT testbed, we find that our methodology while sampling less than 5% of all devices outperforms conventional FedL substantially both in terms of trained model accuracy and required resource utilization.

Original languageEnglish (US)
Title of host publicationINFOCOM 2021 - IEEE Conference on Computer Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9780738112817
DOIs
StatePublished - May 10 2021
Externally publishedYes
Event40th IEEE Conference on Computer Communications, INFOCOM 2021 - Vancouver, Canada
Duration: May 10 2021May 13 2021

Publication series

NameProceedings - IEEE INFOCOM
Volume2021-May
ISSN (Print)0743-166X

Conference

Conference40th IEEE Conference on Computer Communications, INFOCOM 2021
Country/TerritoryCanada
CityVancouver
Period5/10/215/13/21

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Device sampling for heterogeneous federated learning: Theory, algorithms, and implementation'. Together they form a unique fingerprint.

Cite this