Press release: BERKELEY, CA (Thursday, July 18, 2019) - A fundamental norm of science is that its findings are common property of the scientific community and that scientific progress relies on open communication and sharing. Yet all too often, journal editors and reviewers reject papers that have posed important questions and used appropriate methods, but whose results are not statistically significant or are deemed unsurprising or "uninteresting". This publication bias distorts our understanding of the world by sharing only part of the story behind important questions, many of which have real-world policy implications.
To address this, the Journal of Development Economics began piloting Pre-results Review last year in collaboration with the Berkeley Initiative for Transparency in the Social Sciences (UC Berkeley). This new form of peer review, wherein empirical research is reviewed before any results are known, is intended to reward well-designed and well-executed studies regardless of their results. In development economics, pre-results review is another welcome addition to a growing arsenal of transparency- and rigor-enhancing tools such as randomized control trials (RCTs), study pre-registration, and pre-analysis plans.
While the JDE was the first journal in economics to introduce Pre-results Review, more than 200 journals in biology, medicine, political science, psychology, and other disciplines have already done so (in these other disciplines, articles published under this track are called "Registered Reports"). A new post on the World Bank's Development Impact blog shares lessons from the journal's editors, and explains how pre-results review is changing standards at both the journal and the discipline at large. Pre-results Review has helped authors improve the methodological quality of their work and provided an opportunity for earlier recognition - a particularly important incentive for early-career researchers. For editors and reviewers, Pre-results Review has been a useful commitment device for preventing results-based publication bias.
Given high interest among authors and overwhelmingly positive experiences so far, the JDE has made Pre-results Review a permanent track for article submissions. They also invite editors of other economics journals to follow suit: "We believe that further innovation and collective action is critical to fully realizing the potential of Pre-results Review, and more broadly, to continue to bolster the transparency and rigor of economics research" says Andrew Foster, JDE Editor in Chief. To support adoption, they have also made the editorial resources from their pilot available to other journals (and the public). At least one journal, Experimental Economics, is already using them as part of a special issue. However, with rising concerns about pervasive publication bias in economics and other social sciences, many more journals may be ready to use pre-results review to signal their commitment to transparency and credibility.
###
Journal
Journal of Development Economics