TY - JOUR
T1 - Convergence Time Optimization for Federated Learning over Wireless Networks
AU - Chen, Mingzhe
AU - Poor, H. Vincent
AU - Saad, Walid
AU - Cui, Shuguang
N1 - Funding Information:
Manuscript received January 21, 2020; revised July 6, 2020 and September 18, 2020; accepted November 29, 2020. Date of publication December 11, 2020; date of current version April 9, 2021. This work was supported in part by the National Key R&D Program of China under Grant 2018YFB1800800, the Key Area R&D Program of Guangdong Province under Grant 2018B030338001, the Shenzhen Outstanding Talents Training Fund, the Guangdong Research Project under Grant 2017ZT07X152, in part by the U.S. National Science Foundation under Grant CCF-1908308, Grant CCF-0939370, and Grant CCF-1513915, and the Office of Naval Research under Grant N00014-19-1-2621. The associate editor coordinating the review of this article and approving it for publication was L.-C. Wang. (Corresponding author: Shuguang Cui.) Mingzhe Chen is with the Shenzhen Research Institute of Big Data, Shenzhen 518172, China, and also with the Department of Electrical Engineering, Princeton University, Princeton, NJ 08544 USA (e-mail: mingzhec@princeton.edu).
Publisher Copyright:
© 2002-2012 IEEE.
PY - 2021/4
Y1 - 2021/4
N2 - In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.
AB - In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.
KW - Federated learning; wireless resource allocation
KW - artificial neural networks
KW - probabilistic user selection
UR - http://www.scopus.com/inward/record.url?scp=85097925831&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85097925831&partnerID=8YFLogxK
U2 - 10.1109/TWC.2020.3042530
DO - 10.1109/TWC.2020.3042530
M3 - Article
AN - SCOPUS:85097925831
VL - 20
SP - 2457
EP - 2471
JO - IEEE Transactions on Wireless Communications
JF - IEEE Transactions on Wireless Communications
SN - 1536-1276
IS - 4
M1 - 9292468
ER -