Poor research practice suggests true impact of homeopathy may be “substantially” overestimated

Poor research practice suggests that the true impact of homeopathy may be substantially overestimated, finds an analysis of the current body of evidence on the effectiveness of this type of complementary medicine, published online in BMJ Evidence Based Medicine.

Many clinical trials haven’t been registered, with the main outcome changed in a quarter of those that have been. And many remain unpublished. All this indicates “a concerning lack of scientific and ethical standards in the field of homeopathy and a high risk for reporting bias,” say the researchers. 

Homeopathy was developed almost 200 years ago, based on the principle of similarity (‘like cures like’). It remains a popular alternative to conventional medicine in many developed countries, despite its effectiveness being the subject of fierce debate.

The study authors wanted to find out if the published clinical trials might not represent all the scientific studies on homeopathy, but a select few reporting only positive results–a phenomenon known as ‘reporting bias’.

Public clinical trial registries were set up to try and reduce this risk, and since 2008, registration and publication of clinical trial results have been regarded as an ethical, although not mandatory, obligation for researchers.

The study authors therefore set out to: find out how many registered trials assessing homeopathy remain unpublished; whether the primary outcomes of registered trials reflect those actually published; as well as the number of homeopathy trials that had been both registered and published.

They also wanted to assess the impact of any reporting bias on the pooled data analysis of homeopathy trial results, a research method designed to strengthen the evidence base.

They searched major international registries for clinical trials registered up to April 2019, and research databases to track publication of these trials up to April 2021.

They found that since 2002, nearly 38% of registered homeopathy trials remain unpublished, while over half (53%) of published randomised controlled trials haven’t been registered. In all, nearly a third (30%) of randomised controlled trials published during the past 5 years haven’t been registered.

They also found that homeopathy trials were more likely to be registered after they had started (retrospectively) than before they had started (prospective registration). What’s more, a quarter (25%) of published primary outcomes weren’t the same as those originally registered.

The study authors then assessed the potential impact on clinical practice by separately pooling the data from unregistered and registered homeopathy trials. This revealed that unregistered trials tended to report larger treatment effects.

The study authors accept that their searches covered 17 trial registries, so it’s highly likely that they  missed records not covered by these registries. And they pooled the data from homeopathic treatments that weren’t tailored to individual needs, so the findings might not be applicable to personalised treatment.

Nevertheless, the findings “suggest a concerning lack of scientific and ethical standards in the field of homeopathy and a high risk for reporting bias,” they write. 

And they “also indicate that journals publishing homeopathy trials do not adhere to policies by the [International Committee of Medical Journal Editors], which demand that only registered [randomised controlled trials] should be published,” they add.

The poor research practice they found “likely affects the validity of the body of evidence of homeopathic literature and may substantially overestimate the true treatment effect of homeopathic remedies,” they conclude.

Notes for editors
Research: Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis 10.1136/bmjebm-2021-111846
Journal: BMJ Evidence Based Medicine

Funding: None declared

Link to AMS press release labelling system
http://press.psprings.co.uk/AMSlabels.pdf 

Embargoed link to research
http://press.psprings.co.uk/ebm/march/ebm111846.pdf
Public link once embargo lifts
https://ebm.bmj.com/lookup/doi/10.1136/bmjebm-2021-111846

Author contact
Dr Gerald Gartlehner, Department for Evidence-based Medicine and Evaluation, Danube University, Krems, Austria
Tel: + 43 273 289 329 11; cell +43 664 815 3565
Email: Gerald.Gartlehner@donau-uni.ac.at

About the journal
BMJ Evidence Based Medicine is one of more than 70 specialist journals published by BMJ.
https://ebm.bmj.com/

Embargoed 22.30 hours UK (GMT) time Tuesday 15 March 2022

Follow us on Twitter, Facebook, LinkedIn

Need to speak to an expert about a medical issue in the news? Browse our media panel here: bmj.com/company/media-panel/

For further information please contact BMJ media relations team:
mediarelations@bmj.com / +44 (0)7825 118 107 or +44 (0)7388 708 030

For information about how we process personal information and monitor communications please see our privacy policy 

BMJ is a global healthcare knowledge provider with a vision for a healthier world. We share knowledge and expertise to improve healthcare outcomes for all. Explore our tools and resources, and how we are making a difference at bmj.com/company