TY - JOUR
T1 - Successes and Struggles with Computational Reproducibility
T2 - Lessons from the Fragile Families Challenge
AU - Liu, David M.
AU - Salganik, Matthew J.
N1 - Funding Information:
We thank the Board of Advisers of the Fragile Families Challenge. Research reported in this publication was supported by the Russell Sage Foundation and the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under award number P2-CHD047879. Funding for the Fragile Families and Child Wellbeing Study was provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development through grants R01-HD36916, R01-HD39135, and R01-HD40421 and by a consortium of private foundations, including the Robert Wood Johnson Foundation. We thank Caitlin Ahearn, Drew Altschul, Nicole Carnegie, Connor Gilroy, Brian Goode, Seth Green, Jake Hofman, Alex Kindel, Dawn Koffman, Daniel Rigobon, Julia Rohrer, and Janet Xu for helpful feedback on drafts of this article. This article draws on David Liu’s senior thesis for the Department of Computer Science at Princeton University (Liu 2018). The content of this article is solely the responsibility of the authors and does not necessarily represent the views of anyone else.
Publisher Copyright:
© SAGE Publications Inc.. All rights reserved.
PY - 2019
Y1 - 2019
N2 - Reproducibility is fundamental to science, and an important component of reproducibility is computational reproducibility: the ability of a researcher to recreate the results of a published study using the original author’s raw data and code. Although most people agree that computational reproducibility is important, it is still difficult to achieve in practice. In this article, the authors describe their approach to enabling computational reproducibility for the 12 articles in this special issue of Socius about the Fragile Families Challenge. The approach draws on two tools commonly used by professional software engineers but not widely used by academic researchers: software containers (e.g., Docker) and cloud computing (e.g., Amazon Web Services). These tools made it possible to standardize the computing environment around each submission, which will ease computational reproducibility both today and in the future. Drawing on their successes and struggles, the authors conclude with recommendations to researchers and journals.
AB - Reproducibility is fundamental to science, and an important component of reproducibility is computational reproducibility: the ability of a researcher to recreate the results of a published study using the original author’s raw data and code. Although most people agree that computational reproducibility is important, it is still difficult to achieve in practice. In this article, the authors describe their approach to enabling computational reproducibility for the 12 articles in this special issue of Socius about the Fragile Families Challenge. The approach draws on two tools commonly used by professional software engineers but not widely used by academic researchers: software containers (e.g., Docker) and cloud computing (e.g., Amazon Web Services). These tools made it possible to standardize the computing environment around each submission, which will ease computational reproducibility both today and in the future. Drawing on their successes and struggles, the authors conclude with recommendations to researchers and journals.
KW - cloud computing
KW - computational reproducibility
KW - computational social science
KW - open and reproducible research
KW - software containers
UR - http://www.scopus.com/inward/record.url?scp=85092350212&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85092350212&partnerID=8YFLogxK
U2 - 10.1177/2378023119849803
DO - 10.1177/2378023119849803
M3 - Article
C2 - 37309413
AN - SCOPUS:85092350212
SN - 2378-0231
VL - 5
JO - Socius
JF - Socius
M1 - 2378023119849803
ER -