Partial consistency with sparse incidental parameters

The penalized estimation principle is fundamental to high-dimensional problems. In the literature, it has been extensively and successfully applied to various models with only structural parameters. In this paper, we apply this penalization principle to a linear regression model with not only structural parameters but also sparse incidental parameters. For the estimators of the structural parameters, we derive their consistency and asymptotic normality, which reveals an oracle property. However, the penalized estimators for the incidental parameters possess only partial selection consistency, not consistency. This is an interesting partial consistency phenomenon: the structural parameters are consistently estimated while the incidental ones are not. For the structural parameters, also considered is an alternative two-step penalized estimator, which has fewer possible asymptotic distributions and thus is more suitable for statistical inferences. A data-driven approach for selecting a penalty regularization parameter is provided. The finite-sample performance of the penalized estimators for the structural parameters is evaluated by simulations and a data set is analyzed. We also extend the methods and results to the case where the number of the structural parameters diverge but slower than the sample size.