Projection-free online learning

Elad Hazan, Satyen Kale

Research output: Chapter in Book/Report/Conference proceedingConference contribution

52 Scopus citations

Abstract

The computational bottleneck in applying online learning to massive data sets is usually the projection step. We present efficient online learning algorithms that eschew projections in favor of much more efficient linear optimization steps using the Frank-Wolfe technique. We obtain a range of regret bounds for online convex optimization, with better bounds for specific cases such as stochastic online smooth convex optimization. Besides the computational advantage, other desirable features of our algorithms are that they are parameter-free in the stochastic case and produce sparse decisions. We apply our algorithms to computationally intensive applications of collaborative filtering, and show the theoretical improvements to be clearly visible on standard datasets.

Original languageEnglish (US)
Title of host publicationProceedings of the 29th International Conference on Machine Learning, ICML 2012
Pages521-528
Number of pages8
StatePublished - Oct 10 2012
Externally publishedYes
Event29th International Conference on Machine Learning, ICML 2012 - Edinburgh, United Kingdom
Duration: Jun 26 2012Jul 1 2012

Publication series

NameProceedings of the 29th International Conference on Machine Learning, ICML 2012
Volume1

Other

Other29th International Conference on Machine Learning, ICML 2012
CountryUnited Kingdom
CityEdinburgh
Period6/26/127/1/12

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Education

Fingerprint Dive into the research topics of 'Projection-free online learning'. Together they form a unique fingerprint.

  • Cite this

    Hazan, E., & Kale, S. (2012). Projection-free online learning. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012 (pp. 521-528). (Proceedings of the 29th International Conference on Machine Learning, ICML 2012; Vol. 1).