TY - JOUR
T1 - A theory of experimenters
T2 - Robustness, randomization, and balance†
AU - Banerjee, Abhijit V.
AU - Chassang, Sylvain
AU - Montero, Sergio
AU - Snowberg, Erik
N1 - Publisher Copyright:
© 2020 American Economic Association. All rights reserved.
PY - 2020/4
Y1 - 2020/4
N2 - This paper studies the problem of experiment design by an ambiguity-averse decision-maker who trades off subjective expected performance against robust performance guarantees. This framework accounts for real-world experimenters’ preference for randomization. It also clarifies the circumstances in which randomization is optimal: when the available sample size is large and robustness is an important concern. We apply our model to shed light on the practice of rerandomization, used to improve balance across treatment and control groups. We show that rerandomization creates a trade-off between subjective performance and robust performance guarantees. However, robust performance guarantees diminish very slowly with the number of rerandomizations. This suggests that moderate levels of rerandomization usefully expand the set of acceptable compromises between subjective performance and robustness. Targeting a fixed quantile of balance is safer than targeting an absolute balance objective.
AB - This paper studies the problem of experiment design by an ambiguity-averse decision-maker who trades off subjective expected performance against robust performance guarantees. This framework accounts for real-world experimenters’ preference for randomization. It also clarifies the circumstances in which randomization is optimal: when the available sample size is large and robustness is an important concern. We apply our model to shed light on the practice of rerandomization, used to improve balance across treatment and control groups. We show that rerandomization creates a trade-off between subjective performance and robust performance guarantees. However, robust performance guarantees diminish very slowly with the number of rerandomizations. This suggests that moderate levels of rerandomization usefully expand the set of acceptable compromises between subjective performance and robustness. Targeting a fixed quantile of balance is safer than targeting an absolute balance objective.
UR - http://www.scopus.com/inward/record.url?scp=85085298829&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85085298829&partnerID=8YFLogxK
U2 - 10.1257/aer.20171634
DO - 10.1257/aer.20171634
M3 - Article
AN - SCOPUS:85085298829
SN - 0002-8282
VL - 110
SP - 1206
EP - 1230
JO - American Economic Review
JF - American Economic Review
IS - 4
ER -