On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization

Keyphrases

• Expressiveness 100%
• Deep Network 100%
• Conventional Wisdom 50%
• Nonconvex 50%
• Gradient Descent 50%
• Linear Regression 50%
• Tion 50%
• Deep Learning 50%
• Depth Effect 50%
• Regularizer 50%
• Convex Problem 50%
• Learning State 50%
• Linear Neural Network 50%
• Acceleration Schemes 50%
• Acceleration Effect 50%
• Conditioner 50%

Computer Science

• Neural Network 100%
• Speed-up 100%
• Gradient Descent 100%
• Preconditioner 100%
• Conventional Wisdom 100%
• Learning State 100%
• Deep Learning Method 100%

Mathematics

• Expressiveness 100%
• Neural Network 50%
• Convex Problem 50%
• Deep Learning Method 50%
• Linear Regression Analysis 50%