Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning

Xin Yuan, Wei Ni, Ming Ding, Kang Wei, Jun Li, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

While preserving the privacy of federated learning (FL), differential privacy (DP) inevitably degrades the utility (i.e., accuracy) of FL due to model perturbations caused by DP noise added to model updates. Existing studies have considered exclusively noise with persistent root-mean-square amplitude and overlooked an opportunity of adjusting the amplitudes to alleviate the adverse effects of the noise. This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of FL and retain the capability of adjusting the learning performance. Specifically, we propose a geometric series form for the noise amplitude and reveal analytically the dependence of the series on the number of global aggregations and the $(\epsilon,\delta)$ -DP requirement. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. Another important aspect is an upper bound developed for the loss function of a multi-layer perceptron (MLP) trained by FL running the new DP mechanism. Accordingly, the optimal number of global aggregations is obtained, balancing the learning and privacy. Extensive experiments are conducted using MLP, supporting vector machine, and convolutional neural network models on four public datasets. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.

Original languageEnglish (US)
Pages (from-to)1884-1897
Number of pages14
JournalIEEE Transactions on Information Forensics and Security
Volume18
DOIs
StatePublished - 2023
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

Keywords

  • Federated learning
  • convergence analysis
  • differential privacy
  • time-varying noise variance

Fingerprint

Dive into the research topics of 'Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning'. Together they form a unique fingerprint.

Cite this