The Barron Space and the Flow-Induced Function Spaces for Neural Network Models

Weinan E, Chao Ma, Lei Wu

Research output: Contribution to journalArticlepeer-review

54 Scopus citations

Abstract

One of the key issues in the analysis of machine learning models is to identify the appropriate function space and norm for the model. This is the set of functions endowed with a quantity which can control the approximation and estimation errors by a particular machine learning model. In this paper, we address this issue for two representative neural network models: the two-layer networks and the residual neural networks. We define the Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space. For residual neural network models, we construct the so-called flow-induced function space and prove direct and inverse approximation theorems for this space. In addition, we show that the Rademacher complexity for bounded sets under these norms has the optimal upper bounds.

Original languageEnglish (US)
Pages (from-to)369-406
Number of pages38
JournalConstructive Approximation
Volume55
Issue number1
DOIs
StatePublished - Feb 2022

All Science Journal Classification (ASJC) codes

  • Analysis
  • General Mathematics
  • Computational Mathematics

Keywords

  • Approximation
  • Function space
  • Neural network
  • Rademacher complexity

Fingerprint

Dive into the research topics of 'The Barron Space and the Flow-Induced Function Spaces for Neural Network Models'. Together they form a unique fingerprint.

Cite this