A consistent estimator of the expected gradient outerproduct

Shubhendu Trivedi; Jialei Wang; Samory Kpotufe; Gregory Shakhnarovich

A consistent estimator of the expected gradient outerproduct

Shubhendu Trivedi, Jialei Wang, Samory Kpotufe, Gregory Shakhnarovich

Operations Research & Financial Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Scopus citations

Abstract

In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely E_X (Δf(X) Δf(X)^T, is known to recover those directions v ε R^d most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.

Original language	English (US)
Title of host publication	Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014
Editors	Nevin L. Zhang, Jin Tian
Publisher	AUAI Press
Pages	819-828
Number of pages	10
ISBN (Electronic)	9780974903910
State	Published - 2014
Event	30th Conference on Uncertainty in Artificial Intelligence, UAI 2014 - Quebec City, Canada Duration: Jul 23 2014 → Jul 27 2014

Publication series

Name	Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

Other

Other	30th Conference on Uncertainty in Artificial Intelligence, UAI 2014
Country/Territory	Canada
City	Quebec City
Period	7/23/14 → 7/27/14

All Science Journal Classification (ASJC) codes

Artificial Intelligence

Cite this

@inproceedings{840370992a1b4252a0722f1a67d17728,

title = "A consistent estimator of the expected gradient outerproduct",

abstract = "In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX (Δf(X) Δf(X)T, is known to recover those directions v ε Rd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.",

author = "Shubhendu Trivedi and Jialei Wang and Samory Kpotufe and Gregory Shakhnarovich",

year = "2014",

language = "English (US)",

series = "Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014",

publisher = "AUAI Press",

pages = "819--828",

editor = "Zhang, {Nevin L.} and Jin Tian",

booktitle = "Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014",

note = "30th Conference on Uncertainty in Artificial Intelligence, UAI 2014 ; Conference date: 23-07-2014 Through 27-07-2014",

}

Trivedi, S, Wang, J, Kpotufe, S & Shakhnarovich, G 2014, A consistent estimator of the expected gradient outerproduct. in NL Zhang & J Tian (eds), Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014. Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014, AUAI Press, pp. 819-828, 30th Conference on Uncertainty in Artificial Intelligence, UAI 2014, Quebec City, Canada, 7/23/14.

A consistent estimator of the expected gradient outerproduct. / Trivedi, Shubhendu; Wang, Jialei; Kpotufe, Samory et al.
Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014. ed. / Nevin L. Zhang; Jin Tian. AUAI Press, 2014. p. 819-828 (Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A consistent estimator of the expected gradient outerproduct

AU - Trivedi, Shubhendu

AU - Wang, Jialei

AU - Kpotufe, Samory

AU - Shakhnarovich, Gregory

PY - 2014

Y1 - 2014

N2 - In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX (Δf(X) Δf(X)T, is known to recover those directions v ε Rd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.

AB - In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX (Δf(X) Δf(X)T, is known to recover those directions v ε Rd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.

UR - http://www.scopus.com/inward/record.url?scp=84923303869&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923303869&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84923303869

T3 - Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

SP - 819

EP - 828

BT - Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

A2 - Zhang, Nevin L.

A2 - Tian, Jin

PB - AUAI Press

T2 - 30th Conference on Uncertainty in Artificial Intelligence, UAI 2014

Y2 - 23 July 2014 through 27 July 2014

ER -

A consistent estimator of the expected gradient outerproduct

Abstract

Publication series

Other

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this