TY - JOUR
T1 - A simple adjustment for bandwidth snooping
AU - Armstrong, Timothy B.
AU - Kolesár, Michal
N1 - Funding Information:
Acknowledgments. WethankJoshuaAngrist,MatiasCattaneo,VictorChernozhukov,KirillEvdokimov,BoHonoré, Chris Sims, numerous seminar and conference participants, four anonymous referees and the editor for helpful comments and suggestions. We also thank Matias Cattaneo for sharing the Progresa data set. All remaining errors are our own. The research of the first author was supported by National Science Foundation Grant SES-1628939. The research of the second author was supported by National Science Foundation Grant SES-1628878.
Publisher Copyright:
© The Author 2017. Published by Oxford University Press on behalf of The Review of Economic Studies Limited.
PY - 2018/4/1
Y1 - 2018/4/1
N2 - Kernel-based estimators such as local polynomial estimators in regression discontinuity designs are often evaluated at multiple bandwidths as a form of sensitivity analysis. However, if in the reported results, a researcher selects the bandwidth based on this analysis, the associated confidence intervals (CIs) may not have correct coverage, even if the estimator is unbiased. This article proposes a simple adjustment that gives correct coverage in such situations: replace the normal quantile with a critical value that depends only on the kernel and ratio of the maximum and minimum bandwidths the researcher has entertained. We tabulate these critical values and quantify the loss in coverage for conventional CIs. For a range of relevant cases, a conventional 95% CI has coverage between 70% and 90%, and our adjustment amounts to replacing the conventional critical value 1.96 with a number between 2.2 and 2.8. Our results also apply to other settings involving trimmed data, such as trimming to ensure overlap in treatment effect estimation. We illustrate our approach with three empirical applications.
AB - Kernel-based estimators such as local polynomial estimators in regression discontinuity designs are often evaluated at multiple bandwidths as a form of sensitivity analysis. However, if in the reported results, a researcher selects the bandwidth based on this analysis, the associated confidence intervals (CIs) may not have correct coverage, even if the estimator is unbiased. This article proposes a simple adjustment that gives correct coverage in such situations: replace the normal quantile with a critical value that depends only on the kernel and ratio of the maximum and minimum bandwidths the researcher has entertained. We tabulate these critical values and quantify the loss in coverage for conventional CIs. For a range of relevant cases, a conventional 95% CI has coverage between 70% and 90%, and our adjustment amounts to replacing the conventional critical value 1.96 with a number between 2.2 and 2.8. Our results also apply to other settings involving trimmed data, such as trimming to ensure overlap in treatment effect estimation. We illustrate our approach with three empirical applications.
KW - Bandwidth selection
KW - Multiple testing
KW - Nonparametric estimation
KW - Regression discontinuity
UR - http://www.scopus.com/inward/record.url?scp=85044848021&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85044848021&partnerID=8YFLogxK
U2 - 10.1093/restud/rdx051
DO - 10.1093/restud/rdx051
M3 - Article
AN - SCOPUS:85044848021
SN - 0034-6527
VL - 85
SP - 732
EP - 765
JO - Review of Economic Studies
JF - Review of Economic Studies
IS - 2
ER -