Decentralized Stochastic Optimization With Inherent Privacy Protection

Yongqiang Wang, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Decentralized stochastic optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data usually contain sensitive information like user locations, healthcare records, and financial transactions, privacy protection has become an increasingly pressing need in the implementation of decentralized stochastic optimization algorithms. In this article, we propose a decentralized stochastic gradient descent (SGD) algorithm, which is embedded with inherent privacy protection for every participating agent against other participating agents and external eavesdroppers. This proposed algorithm builds in a dynamics based gradient-obfuscation mechanism to enable privacy protection without compromising optimization accuracy, which is in significant difference from differential-privacy based privacy solutions for decentralized optimization that have to trade optimization accuracy for privacy. The dynamics based privacy approach is encryption-free, and hence avoids incurring heavy communication or computation overhead, which is a common problem with encryption based privacy solutions for decentralized stochastic optimization. Besides rigorously characterizing the convergence performance of the proposed decentralized SGD algorithm under both convex objective functions and nonconvex objective functions, we also provide rigorous information-theoretic analysis of its strength of privacy protection. Simulation results for a distributed estimation problem as well as numerical experiments for decentralized learning on a benchmark machine learning dataset confirm the effectiveness of the proposed approach.

Original languageEnglish (US)
Pages (from-to)2293-2308
Number of pages16
JournalIEEE Transactions on Automatic Control
Volume68
Issue number4
DOIs
StatePublished - Apr 1 2023

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Computer Science Applications

Keywords

  • Collaborative machine learning
  • decentralized gradient methods
  • decentralized stochastic optimization
  • privacy protection

Fingerprint

Dive into the research topics of 'Decentralized Stochastic Optimization With Inherent Privacy Protection'. Together they form a unique fingerprint.

Cite this