Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning Without Labels

Satyavrat Wagle, Seyyedali Hosseinalipour, Naji Khosravan, Christopher G. Brinton

Research output: Contribution to journalArticlepeer-review

Abstract

Federated learning (FL) is a popular solution for distributed machine learning (ML). While FL has traditionally been studied for supervised ML tasks, in many applications, it is impractical to assume availability of labeled data across devices. To this end, we develop Cooperative Federated unsupervised Contrastive Learning (CF-CL) to facilitate FL across edge devices with unlabeled datasets. CF-CL employs local device cooperation where either explicit (i.e., raw data) or implicit (i.e., embeddings) information is exchanged through device-to-device (D2D) communications to improve local diversity. Specifically, we introduce a smart information push-pull methodology for data/embedding exchange tailored to FL settings with either soft or strict data privacy restrictions. Information sharing is conducted through a probabilistic importance sampling technique at receivers leveraging a carefully crafted reserve dataset provided by transmitters. In the implicit case, embedding exchange is further integrated into the local ML training at the devices via a regularization term incorporated into the contrastive loss, augmented with a dynamic contrastive margin to adjust the volume of latent space explored. Numerical evaluations demonstrate that CF-CL leads to alignment of latent spaces learned across devices, results in faster and more efficient global model training, and is effective in extreme non-i.i.d. data distribution settings across devices.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Cognitive Communications and Networking
DOIs
StateAccepted/In press - 2024
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence

Keywords

  • Computational modeling
  • Data models
  • Data privacy
  • Information exchange
  • Self-supervised learning
  • Servers
  • Training

Fingerprint

Dive into the research topics of 'Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning Without Labels'. Together they form a unique fingerprint.

Cite this