A simple geometric interpretation of SVM using stochastic adversaries

Roi Livni, Koby Crammer, Amir Globerson

Research output: Contribution to journalConference article

5 Scopus citations

Abstract

We present a minimax framework for classification that considers stochastic adversarial perturbations to the training data. We show that for binary classification it is equivalent to SVM, but with a very natural interpretation of regularization parameter. In the multiclass case, we obtain that our formulation is equivalent to regularizing the hinge loss with the maximum norm of the weight vector (i.e., the two-infinity norm). We test this new regularization scheme and show that it is competitive with the Frobenius regularization commonly used for multiclass SVM. We proceed to analyze various forms of stochastic perturbations and obtain compact optimization problems for the optimal classifiers. Taken together, our results illustrate the advantage of using stochastic perturbations rather than deterministic ones, as well as offer a simple geometric interpretation for SVM optimization.

Original languageEnglish (US)
Pages (from-to)722-730
Number of pages9
JournalJournal of Machine Learning Research
Volume22
StatePublished - Jan 1 2012
Event15th International Conference on Artificial Intelligence and Statistics, AISTATS 2012 - La Palma, Spain
Duration: Apr 21 2012Apr 23 2012

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'A simple geometric interpretation of SVM using stochastic adversaries'. Together they form a unique fingerprint.

  • Cite this