The Reproducibility and Robustness of Economics and Political Science

This systematic and large-scale reproduction effort provides the first evidence of the reproducibility and robustness of economics and political science fields. We reproduced and conducted robustness analysis of 110 articles recently published in leading economics and political science journals. We first find that 84% of published claims can be computationally reproduced. Second, our re-analyses lead to 79% of statistically significant estimates to remain statistically significant in the same direction. Third, the median reproduction’s effect size is 100% the originally published effect size. Fourth, six independent research teams examined 12 pre-specified hypotheses about determinants of reproducibility and robustness. They found a negative relationship between reproducers’ experience and reproducibility, but no relationship between reproducibility and author characteristics or data availability.

Revise & Resubmit, Nature