Gradient descent only converges to minimizers

Jason D. Lee; Max Simchowitz; Michael I. Jordan; Benjamin Recht

Gradient descent only converges to minimizers

Jason D. Lee, Max Simchowitz, Michael I. Jordan, Benjamin Recht

Research output: Contribution to journal › Conference article › peer-review

Abstract

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

Original language	English (US)
Pages (from-to)	1246-1257
Number of pages	12
Journal	Journal of Machine Learning Research
Volume	49
Issue number	June
State	Published - Jun 6 2016
Externally published	Yes
Event	29th Conference on Learning Theory, COLT 2016 - New York, United States Duration: Jun 23 2016 → Jun 26 2016

All Science Journal Classification (ASJC) codes

Software
Artificial Intelligence
Control and Systems Engineering
Statistics and Probability

Keywords

Gradient descent
Local minimum
Non-convex
Saddle points

Cite this

@article{35bdaefcf01346c58310c6e378f45847,

title = "Gradient descent only converges to minimizers",

abstract = "We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.",

keywords = "Gradient descent, Local minimum, Non-convex, Saddle points",

author = "Lee, {Jason D.} and Max Simchowitz and Jordan, {Michael I.} and Benjamin Recht",

note = "Publisher Copyright: {\textcopyright} 2016 J.D. Lee, M. Simchowitz, M.I.J. & B.R. .; 29th Conference on Learning Theory, COLT 2016 ; Conference date: 23-06-2016 Through 26-06-2016",

year = "2016",

month = jun,

day = "6",

language = "English (US)",

volume = "49",

pages = "1246--1257",

journal = "Journal of Machine Learning Research",

issn = "1532-4435",

publisher = "Microtome Publishing",

number = "June",

}

TY - JOUR

T1 - Gradient descent only converges to minimizers

AU - Lee, Jason D.

AU - Simchowitz, Max

AU - Jordan, Michael I.

AU - Recht, Benjamin

PY - 2016/6/6

Y1 - 2016/6/6

N2 - We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

AB - We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

KW - Gradient descent

KW - Local minimum

KW - Non-convex

KW - Saddle points

UR - http://www.scopus.com/inward/record.url?scp=85072246985&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072246985&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85072246985

SN - 1532-4435

VL - 49

SP - 1246

EP - 1257

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

IS - June

T2 - 29th Conference on Learning Theory, COLT 2016

Y2 - 23 June 2016 through 26 June 2016

ER -

Gradient descent only converges to minimizers

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Other files and links

Fingerprint

Cite this