Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchange

Satyavrat Wagle, Seyyedali Hosseinalipour, Naji Khosravan, Mung Chiang, Christopher G. Brinton

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Federated learning (FL) has been recognized as one of the most promising solutions for distributed machine learning (ML). In most of the current literature, FL has been studied for supervised ML tasks, in which edge devices collect labeled data. Nevertheless, in many applications, it is impractical to assume existence of labeled data across devices. To this end, we develop a novel methodology, Cooperative Federated unsupervised Contrastive Learning (CF-CL), for FL across edge devices with unlabeled datasets. CF-CL employs local device cooperation where data are exchanged among devices through device-to-device (D2D) communications to avoid local model bias resulting from non-independent and identically distributed (non-i.i.d.) local datasets. CF-CL introduces a push-pull smart data sharing mechanism tailored to unsupervised FL settings, in which, each device pushes a subset of its local datapoints to its neighbors as reserved datapoints, and pulls a set of datapoints from its neighbors, sampled through a probabilistic importance sampling technique. We demonstrate that CF-CL leads to (i) alignment of unsupervised learned latent spaces across devices, (ii) faster global convergence, allowing for less frequent global model aggregations; and (iii) is effective in extreme non-i.i.d. datasettings across the devices.

Original languageEnglish (US)
Title of host publication2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages492-497
Number of pages6
ISBN (Electronic)9781665435406
DOIs
StatePublished - 2022
Externally publishedYes
Event2022 IEEE Global Communications Conference, GLOBECOM 2022 - Virtual, Online, Brazil
Duration: Dec 4 2022Dec 8 2022

Publication series

Name2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings

Conference

Conference2022 IEEE Global Communications Conference, GLOBECOM 2022
Country/TerritoryBrazil
CityVirtual, Online
Period12/4/2212/8/22

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Renewable Energy, Sustainability and the Environment
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchange'. Together they form a unique fingerprint.

Cite this