Double-anonymous peer review reduces reviewer bias, finds three-year trial

Today (4 April) the British Ecological Society has published the results of a three-year randomised trial comparing double and single-anonymous peer review in the journal Functional Ecology. The findings indicate a reduction in reviewer bias when author identities are anonymised. 

The three-year randomised trial in the journal Functional Ecology, provides the most compressive data yet on the effects of anonymising authors during scholarly journal peer review. 

Double-anonymous peer review, also referred to as double-blind peer review, is where author identities are not disclosed to reviewers. This differs from single-anonymous peer review where reviewers know the identities of the authors, which is more traditional in life science disciplines. 

The study found that when author identities were anonymised and reviewers did not know whose paper they were reviewing (double-anonymous review), peer review outcomes were similar across author demographics. 

In contrast, when author identities were known by reviewers (single-anonymous review), outcomes depended on author demographics. Papers with a first author residing in a higher-income country or a country with a higher average English-proficiency received higher ratings from reviewers and were more likely to be invited for revision or resubmission compared to papers with first authors from a lower income country or country with a lower average English-proficiency. 

Interestingly, anonymising author identities had no effect on gender differences in reviewer ratings or editor decisions. 

Professor Charles Fox, University of Kentucky, lead author of the study and previously the Executive Editor for Functional Ecology said: 

“Our trial provides strong evidence that authors from higher-income and/or English-speaking countries receive significant benefits to being identified to reviewers during the peer review process, and that anonymizing author-identities (e.g., double-blind review) reduces this bias, making the peer review process more equitable. 

“It’s critical for science, and for the scientists involved, that peer review be a fair and unbiased process. The results of this trial will help inform publishers on the best ways to minimise some sources of bias in the publishing process.” 

Based on the strong evidence provided from the trial, the British Ecological Society will begin transitioning its journals to mandatory double-anonymous peer review. This will begin with Functional Ecology, with the other journals published by the BES to follow. 

Andrea Baier, Director of Publishing at the British Ecological Society said: 

“The British Ecological Society is committed to promoting equitable practices in international ecological science. Authors from all over the world submit to our seven journals and it is vitally important that the research we publish is reviewed and selected in the most impartial way, regardless of the authors’ backgrounds.” 

Professor Rob Freckleton, University of Sheffield and Chair of the British Ecological Society’s publications committee said: 

“On behalf of the British Ecological Society, the publications committee supported this important experiment, and from the outset we committed to being led by the results it would produce. We now have the evidence that double-anonymous peer review is an important building block towards greater equity in publishing and we are acting on it.” 

About the trial 

To explore how blinding author identities during peer review reduces biases based on author location or gender, Functional Ecology conducted a large, randomized trial, using real manuscripts submitted from 2019 to 2022. Every research paper submitted to the journal (3739 submissions, 1432 sent for peer review) was randomly assigned to be reviewed double-anonymous or single-anonymous. 

Authors were required to submit their papers to be reviewed with their identities anonymised. Half of submissions were then randomly chosen to have author details added to the papers by adding a title page with information that identified the authors. 

-ENDS-