In a study aiming to replicate laboratory experiments published in high-impact economics journals, researchers reproduced original results in 61% of cases. The work contributes to increasing efforts to demonstrate that scientific findings are reproducible amid continued concern on this matter across disciplines. In recent decades, the reproducibility of results has been questioned in sciences such as medicine, neuroscience and genetics. In economics, concerns have been raised about inflated findings in both empirical and experimental analysis. Here, to contribute data on the latter, Colin Camerer and colleagues replicated 18 experimental studies published in the American Economic Review and the Quarterly Journal of Economics from 2011 through 2014. Four international teams replicated the most important statistically significant finding in each of the 18 studies. All replication and analysis plans were made publicly known on the project website and were also sent to the original authors for verification. To determine what constituted a successful replication, Camerer and colleagues used the same complementary indicators employed in the Reproducibility Project Psychology published in 2015. Ultimately, they were able to reproduce the original results in 11 studies, or 61% of cases. Three more studies were relatively close to being replicated, they said. Their results lead them to conclude that "...replication in this sample of experiments is generally successful, though there is room for improvement." For example, scientists should design and document methods as clearly as possible in anticipation of replications being done in the future.
###
Journal
Science