Characterizing implicit bias in terms of optimization geometry

Suriya Gunasekar, Jason Lee, Daniel Soudry, Nathan Srebro

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

We study the implicit bias of generic optimization methods, including mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing under determined linear regression or separable linear classification problems. We explore the question of whether the specific global minimum (among the many possible global minima) reached by optimization can be characterized in terms of the potential or norm of the optimization geometry, and independently of hypcrparameter choices such as step size and momentum.

Original languageEnglish (US)
Title of host publication35th International Conference on Machine Learning, ICML 2018
EditorsJennifer Dy, Andreas Krause
PublisherInternational Machine Learning Society (IMLS)
Pages2932-2955
Number of pages24
ISBN (Electronic)9781510867963
StatePublished - Jan 1 2018
Externally publishedYes
Event35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden
Duration: Jul 10 2018Jul 15 2018

Publication series

Name35th International Conference on Machine Learning, ICML 2018
Volume4

Other

Other35th International Conference on Machine Learning, ICML 2018
CountrySweden
CityStockholm
Period7/10/187/15/18

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Human-Computer Interaction
  • Software

Fingerprint Dive into the research topics of 'Characterizing implicit bias in terms of optimization geometry'. Together they form a unique fingerprint.

Cite this