Doubts cast on credibility of some published clinical trials

Randomised Controlled Trials (RCTs) are considered the 'gold standard' research method for assessing new medical treatments. But research published in BioMed Central's open access journal Trials shows that the design of a remarkable 93 percent of 2235 so-called RCTs published in some Chinese medical journals during 1994 to 2005 was flawed, casting doubt on the reliability of research that is likely to influence medical decision-makers.

Researchers led by Taixiang Wu of the Chinese Cochrane Centre at Sichuan University, China and Ottawa Hospital Research Institute investigated clinical trials published in China between 1994 and 2005, searching the China National Knowledge Infrastructure (CNKI) electronic database for RCTs on 20 common diseases. To determine how many of these met recognised standards for randomly allocating participants to treatment groups, trained investigators interviewed the first or co-authors of 2235 trial reports by phone.

Less than seven percent of self-described RCTs published in some Chinese medical journals meet criteria for authentic randomisation. The researchers looked at both conventional and traditional Chinese medicine trials, but there was no difference between these in terms of study authenticity rates. However, all RCTs of pre-market drug clinical trial were authentic, and RCTs conducted at hospitals affiliated with medical universities were more likely to be authentic than trials conducted at lower tier level three and level two hospitals. More than half of the trials at university-affiliated hospitals met RCT criteria, which means lower-tier hospital research is the least rigorous in design terms.

"The fact that so many non-RCTs were published as RCTs reflected that peer-review needs to be improved and a Good Practice of Peer Review, including how to identify the authenticity of the study, urgently needs to be developed," says Wu.

Misleading reporting of medical research is not unique to China. Studies labelled as RCTs are more likely to influence health policy-makers meaning falsely reported RCTs have the potential to mislead health care providers, consumers and policy-makers. The results of this study suggest authors of systematic reviews – articles that combine the results of multiple RCTs – need to be aware that RCTs in some Chinese journals may not be RCTs at all.

The approximately 1100 medical journals now active in China are rapidly increasing their output of research reports, including many identified by their authors as RCTs. But these trials present mostly positive results (they favour the treatment being investigated), which can be influenced by inadequate randomisation of patients when designing the study.

Notes to Editors

1. Randomized trials published in some Chinese journals: How many are randomized?
Taixiang Wu, Youping Li, Zhaoxiang Bian, Guanjian Liu and David Moher
Trials (in press)

During embargo, article available here: http://www.trialsjournal.com/imedia/1756122426222937_article.pdf?random=263457

After the embargo, article available at the journal website: http://www.trialsjournal.com/

Please name the journal in any story you write. If you are writing for the web, please link to the article. All articles are available free of charge, according to BioMed Central's open access policy.

Article citation and URL available on request at press@biomedcentral.com on the day of publication.

2. This study was funded by the Chinese Medical Board of New York.

3. Trials is an open access, peer-reviewed, online journal that will encompass all aspects of the performance and findings of randomized controlled trials. Trials will experiment with, and then refine, innovative approaches to improving communication about trials. We are keen to move beyond publishing traditional trial results articles (although these will be included). We believe this represents an exciting opportunity to advance the science and reporting of trials. Prior to 2006, Trials was published as Current Controlled Trials in Cardiovascular Medicine (CCTCVM). All published CCTCVM articles are available via the Trials website and citations to CCTCVM article URLs will continue to be supported.

4. BioMed Central (http://www.biomedcentral.com/) is an STM (Science, Technology and Medicine) publisher which has pioneered the open access publishing model. All peer-reviewed research articles published by BioMed Central are made immediately and freely accessible online, and are licensed to allow redistribution and reuse. BioMed Central is part of Springer Science+Business Media, a leading global publisher in the STM sector.