TY - GEN
T1 - Federated Learning beyond the Star
T2 - 2021 IEEE Global Communications Conference, GLOBECOM 2021
AU - Lin, Frank Po Chen
AU - Hosseinalipour, Seyyedali
AU - Azam, Sheikh Shams
AU - Brinton, Christopher G.
AU - Michelusi, Nicolo
N1 - Funding Information:
An extended version of this paper is published in IEEE JSAC [1]. The work of N. Michelusi was supported in part by NSF under grants CNS-1642982 and CNS-2129015. C. Brinton and F. Lin were supported in part by ONR under grant N00014-21-1-2472.
Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Federated learning has emerged as a popular technique for distributing model training across the network edge. Its learning architecture is conventionally a star topology be-tween the devices and a central server. In this paper, we propose two timescale hybrid federated learning (TT-Hf), which migrates to a more distributed topology via device-to-device (D2D) communications. In TT-HF, local model training occurs at devices via successive gradient iterations, and the synchronization process occurs at two timescales: (i) macro-scale, where global aggregations are carried out via device-server interactions, and (ii) micro-scale, where local aggregations are carried out via D2D cooperative consensus formation in different device clusters. Our theoretical analysis reveals how device, cluster, and network-level parameters affect the convergence of TT-HF, and leads to a set of conditions under which a convergence rate of O(1/t) is guaranteed. Experimental results demonstrate the improvements in convergence and utilization that can be obtained by TT-HF over state-of-the-art federated learning baselines.
AB - Federated learning has emerged as a popular technique for distributing model training across the network edge. Its learning architecture is conventionally a star topology be-tween the devices and a central server. In this paper, we propose two timescale hybrid federated learning (TT-Hf), which migrates to a more distributed topology via device-to-device (D2D) communications. In TT-HF, local model training occurs at devices via successive gradient iterations, and the synchronization process occurs at two timescales: (i) macro-scale, where global aggregations are carried out via device-server interactions, and (ii) micro-scale, where local aggregations are carried out via D2D cooperative consensus formation in different device clusters. Our theoretical analysis reveals how device, cluster, and network-level parameters affect the convergence of TT-HF, and leads to a set of conditions under which a convergence rate of O(1/t) is guaranteed. Experimental results demonstrate the improvements in convergence and utilization that can be obtained by TT-HF over state-of-the-art federated learning baselines.
UR - http://www.scopus.com/inward/record.url?scp=85119159336&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85119159336&partnerID=8YFLogxK
U2 - 10.1109/GLOBECOM46510.2021.9685456
DO - 10.1109/GLOBECOM46510.2021.9685456
M3 - Conference contribution
AN - SCOPUS:85119159336
T3 - 2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings
BT - 2021 IEEE Global Communications Conference, GLOBECOM 2021 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 7 December 2021 through 11 December 2021
ER -