A priori estimates of the population risk for two-layer neural networks

E. Weinan, Chao Ma, Lei Wu

Research output: Contribution to journalArticlepeer-review

42 Scopus citations

Abstract

New estimates for the population risk are established for two-layer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the over-parametrized regime when the network size is much larger than the size of the dataset. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model, in contrast with most existing results which are a posteriori in nature. Using these a priori estimates, we provide a perspective for understanding why two-layer neural networks perform better than the related kernel methods.

Original languageEnglish (US)
Pages (from-to)1407-1425
Number of pages19
JournalCommunications in Mathematical Sciences
Volume17
Issue number5
DOIs
StatePublished - 2019

All Science Journal Classification (ASJC) codes

  • Mathematics(all)
  • Applied Mathematics

Keywords

  • A priori estimate
  • Barron space
  • Population risk
  • Rademacher complexity
  • Two-layer neural network

Fingerprint

Dive into the research topics of 'A priori estimates of the population risk for two-layer neural networks'. Together they form a unique fingerprint.

Cite this