Model-Based Reinforcement Learning for Quantized Federated Learning Performance Optimization

Nuocheng Yang, Sihua Wang, Mingzhe Chen, Christopher G. Brinton, Changchuan Yin, Walid Saad, Shuguang Cui

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which, in turn, aggregates them into a quantized global model and synchronizes the devices. With the goal of jointly determining the set of participating devices in each training iteration and the bitwidths employed at the devices, we pose an optimization problem for minimizing the training loss of quantized FL under a device sampling budget and delay requirement. Our analytical results show that the improvement of FL training loss between two consecutive iterations depends on not only the device selection and quantization scheme, but also on several parameters inherent to the model being learned. As a result, we propose, a model-based reinforcement learning (RL) method to optimize action selection over iterations. Compared to model-free RL, the proposed approach leverages the derived mathematical characterization of the FL training process to discover an effective device selection and quantization scheme without imposing additional device communication overhead. Numerical evaluations show that the proposed FL framework can achieve the same classification performance while reducing the number of training iterations needed for convergence by 20% compared to model-free RL-based FL.

Original languageEnglish (US)
Title of host publication2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages5063-5068
Number of pages6
ISBN (Electronic)9781665435406
DOIs
StatePublished - 2022
Externally publishedYes
Event2022 IEEE Global Communications Conference, GLOBECOM 2022 - Virtual, Online, Brazil
Duration: Dec 4 2022Dec 8 2022

Publication series

Name2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings

Conference

Conference2022 IEEE Global Communications Conference, GLOBECOM 2022
Country/TerritoryBrazil
CityVirtual, Online
Period12/4/2212/8/22

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Renewable Energy, Sustainability and the Environment
  • Safety, Risk, Reliability and Quality

Keywords

  • Bitwidth federated learning
  • FL training loss optimization
  • model-based reinforcement learning

Fingerprint

Dive into the research topics of 'Model-Based Reinforcement Learning for Quantized Federated Learning Performance Optimization'. Together they form a unique fingerprint.

Cite this