RADEMACHER COMPLEXITY AND THE GENERALIZATION ERROR OF RESIDUAL NETWORKS

WEINAN E, CHAO MA, QINGCAN WANG

Research output: Contribution to journalArticlepeer-review

Abstract

. Sharp bounds for the Rademacher complexity and the generalization error are derived for the residual network model. The Rademacher complexity bound has no explicit dependency on the depth of the network, while the generalization bounds are comparable to the Monte Carlo error rates, suggesting that they are nearly optimal in the high dimensional setting. These estimates are achieved by constraining the hypothesis space with an appropriately defined path norm such that the constrained space is large enough for the approximation error rates to be optimal and small enough for the estimation error rates to be optimal at the same time. Comparisons are made with other norm-based bounds.

Original languageEnglish (US)
Pages (from-to)1755-1774
Number of pages20
JournalCommunications in Mathematical Sciences
Volume18
Issue number6
DOIs
StatePublished - 2020

All Science Journal Classification (ASJC) codes

  • Mathematics(all)
  • Applied Mathematics

Keywords

  • a priori estimate
  • residual network
  • weighted path norm

Fingerprint Dive into the research topics of 'RADEMACHER COMPLEXITY AND THE GENERALIZATION ERROR OF RESIDUAL NETWORKS'. Together they form a unique fingerprint.

Cite this