TY - GEN

T1 - A consistent estimator of the expected gradient outerproduct

AU - Trivedi, Shubhendu

AU - Wang, Jialei

AU - Kpotufe, Samory K.

AU - Shakhnarovich, Gregory

PY - 2014/1/1

Y1 - 2014/1/1

N2 - In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX (Δf(X) Δf(X)T, is known to recover those directions v ε Rd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.

AB - In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely EX (Δf(X) Δf(X)T, is known to recover those directions v ε Rd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.

UR - http://www.scopus.com/inward/record.url?scp=84923303869&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923303869&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84923303869

T3 - Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

SP - 819

EP - 828

BT - Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, UAI 2014

A2 - Zhang, Nevin L.

A2 - Tian, Jin

PB - AUAI Press

T2 - 30th Conference on Uncertainty in Artificial Intelligence, UAI 2014

Y2 - 23 July 2014 through 27 July 2014

ER -