Embedding hard learning problems into gaussian space

Adam Klivans, Pravesh Kothari

Research output: Chapter in Book/Report/Conference proceedingConference contribution

28 Scopus citations

Abstract

with respect to the Gaussian distribution. We reduce from the problem of learning sparse parities with noise with respect to the uniform distribution on the hypercube (sparse LPN), a notoriously hard problem in theoretical computer science and show that any algorithm for agnostically learning halfspaces requires nΩ(log (1/ε)) time under the assumption that κ-sparse LPN requires nΩ(κ) time, ruling out a polynomial time algorithm for the problem. As far as we are aware, this is the first representation-independent hardness result for supervised learning when the underlying distribution is restricted to be a Gaussian. We also show that the problem of agnostically learning sparse polynomials with respect to the Gaussian distribution in polynomial time is as hard as PAC learning DNFs on the uniform distribution in polynomial time. This complements the surprising result of Andoni et. al. [1] who show that sparse polynomials are learnable under random Gaussian noise in polynomial time. Taken together, these results show the inherent difficulty of designing supervised learning algorithms in Euclidean space even in the presence of strong distributional assumptions. Our results use a novel embedding of random labeled examples from the uniform distribution on the Boolean hypercube into random labeled examples from the Gaussian distribution that allows us to relate the hardness of learning problems on two different domains and distributions.

Original languageEnglish (US)
Title of host publicationLeibniz International Proceedings in Informatics, LIPIcs
EditorsKlaus Jansen, Jose D. P. Rolim, Nikhil R. Devanur, Cristopher Moore
PublisherSchloss Dagstuhl- Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing
Pages793-809
Number of pages17
ISBN (Electronic)9783939897743
DOIs
StatePublished - Sep 1 2014
Externally publishedYes
Event17th International Workshop on Approximation Algorithms for Combinatorial Optimization Problems, APPROX 2014 and the 18th International Workshop on Randomization and Computation, RANDOM 2014 - Barcelona, Spain
Duration: Sep 4 2014Sep 6 2014

Publication series

NameLeibniz International Proceedings in Informatics, LIPIcs
Volume28
ISSN (Print)1868-8969

Other

Other17th International Workshop on Approximation Algorithms for Combinatorial Optimization Problems, APPROX 2014 and the 18th International Workshop on Randomization and Computation, RANDOM 2014
Country/TerritorySpain
CityBarcelona
Period9/4/149/6/14

All Science Journal Classification (ASJC) codes

  • Software

Keywords

  • Agnostic learning
  • Distribution-specific hardness of learning
  • Gaussian space
  • Halfspacelearning

Fingerprint

Dive into the research topics of 'Embedding hard learning problems into gaussian space'. Together they form a unique fingerprint.

Cite this