A consistent estimator of the expected gradient outerproduct

Shubhendu Trivedi, Jialei Wang, Samory K. Kpotufe, Gregory Shakhnarovich

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX (Δf(X) Δf(X)T, is known to recover those directions v ε Rd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.

Original languageEnglish (US)
Title of host publicationUncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014
EditorsNevin L. Zhang, Jin Tian
PublisherAUAI Press
Pages819-828
Number of pages10
ISBN (Electronic)9780974903910
StatePublished - Jan 1 2014
Event30th Conference on Uncertainty in Artificial Intelligence, UAI 2014 - Quebec City, Canada
Duration: Jul 23 2014Jul 27 2014

Publication series

NameUncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

Other

Other30th Conference on Uncertainty in Artificial Intelligence, UAI 2014
CountryCanada
CityQuebec City
Period7/23/147/27/14

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Cite this

Trivedi, S., Wang, J., Kpotufe, S. K., & Shakhnarovich, G. (2014). A consistent estimator of the expected gradient outerproduct. In N. L. Zhang, & J. Tian (Eds.), Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014 (pp. 819-828). (Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014). AUAI Press.