TY - GEN
T1 - Generalization error of deep neural networks
T2 - 12th International Conference on Sampling Theory and Applications, SampTA 2017
AU - Sokolić, Jure
AU - Giryes, Raja
AU - Sapiro, Guillermo
AU - Rodrigues, Miguel R.D.
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/9/1
Y1 - 2017/9/1
N2 - Understanding the generalization properties of deep learning models is critical for their successful usage in many applications, especially in the regimes where the number of training samples is limited. We study the generalization properties of deep neural networks (DNNs) via the Jacobian matrix of the network. Our analysis is general to arbitrary network structures, types of non-linearities and pooling operations. We show that bounding the spectral norm of the Jacobian matrix in the network reduces the generalization error. In addition, we tie this error to the invariance in the data and the network. Experiments on the MNIST and ImageNet datasets support these findings. This short paper summarizes our generalization error theorems for DNNs and for general invariant classifiers [1], [2].
AB - Understanding the generalization properties of deep learning models is critical for their successful usage in many applications, especially in the regimes where the number of training samples is limited. We study the generalization properties of deep neural networks (DNNs) via the Jacobian matrix of the network. Our analysis is general to arbitrary network structures, types of non-linearities and pooling operations. We show that bounding the spectral norm of the Jacobian matrix in the network reduces the generalization error. In addition, we tie this error to the invariance in the data and the network. Experiments on the MNIST and ImageNet datasets support these findings. This short paper summarizes our generalization error theorems for DNNs and for general invariant classifiers [1], [2].
UR - http://www.scopus.com/inward/record.url?scp=85031704288&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85031704288&partnerID=8YFLogxK
U2 - 10.1109/SAMPTA.2017.8024476
DO - 10.1109/SAMPTA.2017.8024476
M3 - Conference contribution
AN - SCOPUS:85031704288
T3 - 2017 12th International Conference on Sampling Theory and Applications, SampTA 2017
SP - 147
EP - 151
BT - 2017 12th International Conference on Sampling Theory and Applications, SampTA 2017
A2 - Anbarjafari, Gholamreza
A2 - Kivinukk, Andi
A2 - Tamberg, Gert
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 3 July 2017 through 7 July 2017
ER -