SplitGP: Achieving Both Generalization and Personalization in Federated Learning

Dong Jun Han, Do Yeon Kim, Minseok Choi, Christopher G. Brinton, Jaekyun Moon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

A fundamental challenge to providing edge-AI services is the need for a machine learning (ML) model that achieves personalization (i.e., to individual clients) and generalization (i.e., to unseen data) properties concurrently. Existing techniques in federated learning (FL) have encountered a steep tradeoff between these objectives and impose large computational requirements on edge devices during training and inference. In this paper, we propose SplitGP, a new split learning solution that can simultaneously capture generalization and personalization capabilities for efficient inference across resource-constrained clients (e.g., mobile/IoT devices). Our key idea is to split the full ML model into client-side and server-side components, and impose different roles to them: the client-side model is trained to have strong personalization capability optimized to each client's main task, while the server-side model is trained to have strong generalization capability for handling all clients' out-of-distribution tasks. We analytically characterize the convergence behavior of SplitGP, revealing that all client models approach stationary points asymptotically. Further, we analyze the inference time in SplitGP and provide bounds for determining model split ratios. Experimental results show that SplitGP outperforms existing baselines by wide margins in inference time and test accuracy for varying amounts of out-of-distribution samples.

Original languageEnglish (US)
Title of host publicationINFOCOM 2023 - IEEE Conference on Computer Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350334142
DOIs
StatePublished - 2023
Externally publishedYes
Event42nd IEEE International Conference on Computer Communications, INFOCOM 2023 - Hybrid, New York City, United States
Duration: May 17 2023May 20 2023

Publication series

NameProceedings - IEEE INFOCOM
Volume2023-May
ISSN (Print)0743-166X

Conference

Conference42nd IEEE International Conference on Computer Communications, INFOCOM 2023
Country/TerritoryUnited States
CityHybrid, New York City
Period5/17/235/20/23

All Science Journal Classification (ASJC) codes

  • General Computer Science
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'SplitGP: Achieving Both Generalization and Personalization in Federated Learning'. Together they form a unique fingerprint.

Cite this