TY - GEN
T1 - Model-Based Reinforcement Learning for Quantized Federated Learning Performance Optimization
AU - Yang, Nuocheng
AU - Wang, Sihua
AU - Chen, Mingzhe
AU - Brinton, Christopher G.
AU - Yin, Changchuan
AU - Saad, Walid
AU - Cui, Shuguang
N1 - Funding Information:
This work was supported in part by the National Natural Science Foundation of China under Grants 61871041, 61629101 and 61671086, in part by Beijing Natural Science Foundation-Haidian Original Innovation Foundation (L192003), in part by office of Naval Research grant N00014-21-1-2472 and National Science Foundation grant CNS-2146171, in part by the U.S. National Science Foundation under Grant CNS-2114267, in part by the Basic Research Project No. HZQB-KCZYZ-2021067 of Hetao Shenzhen-HK ST Cooperation Zone, in part by Guangdong Research Projects No. 2017ZT07X152 and No. 2019CX01X104, and in part by the Guangdong Provincial Key Laboratory of Future Networks of Intelligence (Grant No. 2022B1212010001).
Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which, in turn, aggregates them into a quantized global model and synchronizes the devices. With the goal of jointly determining the set of participating devices in each training iteration and the bitwidths employed at the devices, we pose an optimization problem for minimizing the training loss of quantized FL under a device sampling budget and delay requirement. Our analytical results show that the improvement of FL training loss between two consecutive iterations depends on not only the device selection and quantization scheme, but also on several parameters inherent to the model being learned. As a result, we propose, a model-based reinforcement learning (RL) method to optimize action selection over iterations. Compared to model-free RL, the proposed approach leverages the derived mathematical characterization of the FL training process to discover an effective device selection and quantization scheme without imposing additional device communication overhead. Numerical evaluations show that the proposed FL framework can achieve the same classification performance while reducing the number of training iterations needed for convergence by 20% compared to model-free RL-based FL.
AB - This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which, in turn, aggregates them into a quantized global model and synchronizes the devices. With the goal of jointly determining the set of participating devices in each training iteration and the bitwidths employed at the devices, we pose an optimization problem for minimizing the training loss of quantized FL under a device sampling budget and delay requirement. Our analytical results show that the improvement of FL training loss between two consecutive iterations depends on not only the device selection and quantization scheme, but also on several parameters inherent to the model being learned. As a result, we propose, a model-based reinforcement learning (RL) method to optimize action selection over iterations. Compared to model-free RL, the proposed approach leverages the derived mathematical characterization of the FL training process to discover an effective device selection and quantization scheme without imposing additional device communication overhead. Numerical evaluations show that the proposed FL framework can achieve the same classification performance while reducing the number of training iterations needed for convergence by 20% compared to model-free RL-based FL.
KW - Bitwidth federated learning
KW - FL training loss optimization
KW - model-based reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85138878780&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85138878780&partnerID=8YFLogxK
U2 - 10.1109/GLOBECOM48099.2022.10001466
DO - 10.1109/GLOBECOM48099.2022.10001466
M3 - Conference contribution
AN - SCOPUS:85138878780
T3 - 2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings
SP - 5063
EP - 5068
BT - 2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 IEEE Global Communications Conference, GLOBECOM 2022
Y2 - 4 December 2022 through 8 December 2022
ER -