Communication-Efficient Accurate Statistical Estimation

Jianqing Fan, Yongyi Guo, Kaizheng Wang

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

When the data are stored in a distributed manner, direct applications of traditional statistical inference procedures are often prohibitive due to communication costs and privacy concerns. This article develops and investigates two communication-efficient accurate statistical estimators (CEASE), implemented through iterative algorithms for distributed optimization. In each iteration, node machines carry out computation in parallel and communicate with the central processor, which then broadcasts aggregated information to node machines for new updates. The algorithms adapt to the similarity among loss functions on node machines, and converge rapidly when each node machine has large enough sample size. Moreover, they do not require good initialization and enjoy linear converge guarantees under general conditions. The contraction rate of optimization errors is presented explicitly, with dependence on the local sample size unveiled. In addition, the improved statistical accuracy per iteration is derived. By regarding the proposed method as a multistep statistical estimator, we show that statistical efficiency can be achieved in finite steps in typical statistical applications. In addition, we give the conditions under which the one-step CEASE estimator is statistically efficient. Extensive numerical experiments on both synthetic and real data validate the theoretical results and demonstrate the superior performance of our algorithms.

Original languageEnglish (US)
JournalJournal of the American Statistical Association
DOIs
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Keywords

  • Communication efficiency
  • Distributed statistical estimation
  • Multi-round algorithms
  • Penalized likelihood

Fingerprint

Dive into the research topics of 'Communication-Efficient Accurate Statistical Estimation'. Together they form a unique fingerprint.

Cite this