DEEP NEURAL NETWORKS FOR NONPARAMETRIC INTERACTION MODELS WITH DIVERGING DIMENSION

Sohom Bhattacharya, Jianqing Fan, Debarghya Mukherjee

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Deep neural networks have achieved tremendous success due to their representation power and adaptation to low-dimensional structures. Their potential for estimating structured regression functions has been recently established in the literature. However, most of the studies require the input dimension to be fixed, and consequently, they ignore the effect of dimension on the rate of convergence and hamper their applications to modern big data with high dimensionality. In this paper, we bridge this gap by analyzing a k-way nonparametric interaction model in both growing dimension scenarios (d grows with n but at a slower rate) and in high dimension (d > n). In the latter case, sparsity assumptions and associated regularization are required to obtain optimal convergence rates. A new challenge in diverging dimension setting is in calculation mean-square error; the covariance terms among estimated additive components are an order of magnitude larger than those of the variances and can deteriorate statistical properties without proper care. We introduce a critical debiasing technique to amend the problem. We show that under certain standard assumptions, debiased deep neural networks achieve a minimax optimal rate both in terms of (n, d). Our proof techniques rely crucially on a novel debiasing technique that makes the covariances of additive components negligible in the mean-square error calculation. In addition, we establish the matching lower bounds.

Original languageEnglish (US)
Pages (from-to)2738-2766
Number of pages29
JournalAnnals of Statistics
Volume52
Issue number6
DOIs
StatePublished - Dec 2024
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Keywords

  • Deep neural networks
  • high-dimensional statistics
  • minimax rate
  • nonparametric interaction model
  • sparse nonparametric components

Fingerprint

Dive into the research topics of 'DEEP NEURAL NETWORKS FOR NONPARAMETRIC INTERACTION MODELS WITH DIVERGING DIMENSION'. Together they form a unique fingerprint.

Cite this