Personalized Federated Learning With Differential Privacy and Convergence Guarantee

Kang Wei, Jun Li, Chuan Ma, Ming Ding, Wen Chen, Jun Wu, Meixia Tao, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Personalized federated learning (PFL), as a novel federated learning (FL) paradigm, is capable of generating personalized models for heterogenous clients. Combined with a meta-learning mechanism, PFL can further improve the convergence performance with few-shot training. However, meta-learning based PFL has two stages of gradient descent in each local training round, therefore posing a more serious challenge in information leakage. In this paper, we propose a differential privacy (DP) based PFL (DP-PFL) framework and analyze its convergence performance. Specifically, we first design a privacy budget allocation scheme for inner and outer update stages based on the Rényi DP composition theory. Then, we develop two convergence bounds for the proposed DP-PFL framework under convex and non-convex loss function assumptions, respectively. Our developed convergence bounds reveal that 1) there is an optimal size of the DP-PFL model that can achieve the best convergence performance for a given privacy level, and 2) there is an optimal tradeoff among the number of communication rounds, convergence performance and privacy budget. Evaluations on various real-life datasets demonstrate that our theoretical results are consistent with experimental results. The derived theoretical results can guide the design of various DP-PFL algorithms with configurable tradeoff requirements on the convergence performance and privacy levels.

Original languageEnglish (US)
Pages (from-to)4488-4503
Number of pages16
JournalIEEE Transactions on Information Forensics and Security
Volume18
DOIs
StatePublished - 2023
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

Keywords

  • Federated learning
  • convergence analysis
  • differential privacy
  • meta-learning

Fingerprint

Dive into the research topics of 'Personalized Federated Learning With Differential Privacy and Convergence Guarantee'. Together they form a unique fingerprint.

Cite this