AI can fake peer reviews and escape detection, study finds

Large language models (LLMs) like ChatGPT can be used to write convincing but biased peer reviews that are nearly impossible to distinguish from human writing, a new study reveals. This poses a serious threat to the integrity of scientific publishing, where peer review is the critical process for vetting research quality and accuracy.

In a study evaluating the risks of AI in academic publishing, a team of researchers from China tasked the AI model Claude with reviewing 20 real cancer research manuscripts. To closely simulate the real-world peer review process, they used the initial manuscripts submitted for review—sourced from the journal eLife's transparent peer review model—rather than the final, published versions of the articles. The AI was instructed to perform several tasks a human reviewer might: write a standard review, recommend a paper for rejection, and even request that authors cite specific articles—some of which were completely unrelated to the research topic.

The results were alarming. The researchers found that popular AI detection tools were largely ineffective, with one detector flagging over 80% of the AI-generated reviews as "human-written." While the AI's standard reviews often lacked the depth of an expert, it excelled at generating persuasive rejection comments and fabricating plausible-sounding reasons to cite irrelevant studies.

"We were surprised by how easily the LLM could generate convincing rejection comments and seemingly reasonable requests to cite unrelated papers," explains Peng Luo, one of the study's corresponding authors from the Department of Oncology at Zhujiang Hospital, Southern Medical University. "This creates a serious risk. Malicious reviewers could use this technology to unfairly reject good research or to manipulate citation numbers for their own benefit. The system is built on trust, and this technology can break that trust."

However, the researchers also found a potential upside. The same AI was effective at writing strong rebuttals to these unreasonable citation requests, offering a new tool for authors to defend their work against unfair criticism.

The authors urge the academic community to establish clear guidelines and new oversight mechanisms to ensure AI is used responsibly to support, not undermine, scientific integrity.