Representation formulas and pointwise properties for Barron functions

E. Weinan, Stephan Wojtowytsch

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae. In two cases, we describe the space explicitly up to isomorphism. Using a convenient representation, we study the pointwise properties of two-layer networks and show that functions whose singular set is fractal or curved (for example distance functions from smooth submanifolds) cannot be represented by infinitely wide two-layer networks with finite path-norm. We use this structure theorem to show that the only C1-diffeomorphisms which preserve Barron space are affine. Furthermore, we show that every Barron function can be decomposed as the sum of a bounded and a positively one-homogeneous function and that there exist Barron functions which decay rapidly at infinity and are globally Lebesgue-integrable. This result suggests that two-layer neural networks may be able to approximate a greater variety of functions than commonly believed.

Original languageEnglish (US)
Article number46
JournalCalculus of Variations and Partial Differential Equations
Volume61
Issue number2
DOIs
StatePublished - Apr 2022

All Science Journal Classification (ASJC) codes

  • Analysis
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Representation formulas and pointwise properties for Barron functions'. Together they form a unique fingerprint.

Cite this