Ultrahigh Dimensional Feature Selection: Beyond the Linear Model

Jianqing Fan, Richard Samworth, Yichao Wu

Research output: Contribution to journalArticle

228 Scopus citations

Abstract

Variable selection in high-dimensional space characterizes many contemporary problems in scientific discovery and decision making. Many frequently-used techniques are based on independence screening; examples include correlation ranking (Fan & Lv, 2008) or feature selection using a twosample t-test in high-dimensional classification (Tibshirani et al., 2003). Within the context of the linear model, Fan & Lv (2008) showed that this simple correlation ranking possesses a sure independence screening property under certain conditions and that its revision, called iteratively sure independent screening (ISIS), is needed when the features are marginally unrelated but jointly related to the response variable. In this paper, we extend ISIS, without explicit definition of residuals, to a general pseudo-likelihood framework, which includes generalized linear models as a special case. Even in the least-squares setting, the new method improves ISIS by allowing feature deletion in the iterative process. Our technique allows us to select important features in high-dimensional classification where the popularly used two-sample Mnethod fails. A new technique is introduced to reduce the false selection rate in the feature screening stage. Several simulated and two real data examples are presented to illustrate the methodology.

Original languageEnglish (US)
Pages (from-to)2013-2038
Number of pages26
JournalJournal of Machine Learning Research
Volume10
StatePublished - Nov 18 2009

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Statistics and Probability
  • Artificial Intelligence

Keywords

  • Classification
  • Feature screening
  • Feature selection
  • Generalized linear models
  • Robust regression

Fingerprint Dive into the research topics of 'Ultrahigh Dimensional Feature Selection: Beyond the Linear Model'. Together they form a unique fingerprint.

  • Cite this