Faster rates for the frank-Wolfe method over strongly-convex sets

Dan Garber, Elad Hazan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

38 Scopus citations

Abstract

The Frank-Wolfe method (a.k.a. conditional gradient algorithm) for smooth optimization has regained much interest in recent years in the context of large scale optimization and machine learning. A key advantage of the method is that it avoids projections - the computational bottleneck in many applications - replacing it by a linear optimization step. Despite this advantage, the known convergence rates of the FW method fall behind standard first order methods for most settings of interest. It is an active line of research to derive faster linear optimization-based algorithms for various settings of convex optimization. In this paper we consider the special case of optimization over strongly convex sets, for which we prove that the vanila FW method converges at a rate of 1/t2. This gives a quadratic improvement in convergence rate compared to the general case, in which convergence is of the order t, and known to be tight. We show that various balls induced by ℓp norms, Schatten norms and group norms are strongly convex on one hand and on the other hand, linear optimization over these sets is straightforward and admits a closed-form solution. We further show how several previous fast-rate results for the FW method follow easily from our analysis.

Original languageEnglish (US)
Title of host publication32nd International Conference on Machine Learning, ICML 2015
EditorsFrancis Bach, David Blei
PublisherInternational Machine Learning Society (IMLS)
Pages541-549
Number of pages9
ISBN (Electronic)9781510810587
StatePublished - Jan 1 2015
Event32nd International Conference on Machine Learning, ICML 2015 - Lile, France
Duration: Jul 6 2015Jul 11 2015

Publication series

Name32nd International Conference on Machine Learning, ICML 2015
Volume1

Other

Other32nd International Conference on Machine Learning, ICML 2015
CountryFrance
CityLile
Period7/6/157/11/15

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Faster rates for the frank-Wolfe method over strongly-convex sets'. Together they form a unique fingerprint.

  • Cite this

    Garber, D., & Hazan, E. (2015). Faster rates for the frank-Wolfe method over strongly-convex sets. In F. Bach, & D. Blei (Eds.), 32nd International Conference on Machine Learning, ICML 2015 (pp. 541-549). (32nd International Conference on Machine Learning, ICML 2015; Vol. 1). International Machine Learning Society (IMLS).