Thirty-eight percent of the Nature papers and a quarter of the BMJ articles studied contained at least one statistical error, according to Emili García-Berthou, a lecturer on biostatistics at University of Girona, Spain, and Carles Alcaraz. In total, more than 11% of the statistical results published in the two journals during 2001 were incongruent.
"Our findings confirm that the quality of research and scientific papers needs improvement and should be more carefully checked and evaluated in these days of high publication pressure," write the authors.
The errors seen could have been caused by transcription or typesetting errors, for example if a repeated zero was omitted. Alternatively, researchers may have rounded up figures incorrectly.
The researchers showed that some numbers, four and nine, were seen less often than would be expected at the end of a given test statistic or P-value, suggesting that researchers were rounding up numbers incorrectly, possibly so that they looked 'neater'. For example study authors might round up 2.38 to 2.5 rather than 2.4.
"Although these kinds of errors may leave the conclusions of a study unchanged, they are indicative of poor practice," say the researchers. "Our concern is that these kinds of errors are probably present in all numerical results and all steps of scientific research, with potentially important practical consequences."
The researchers suggest that one way to minimise the effect of these errors would be for published authors to make their raw data freely available on the Internet. This would allow other researchers to check for themselves whether the results of the study are correct and the conclusions justified. "Also, fraud and sloppiness may be more easily detected," they say.
This press release is based on the following article:
Incongruence between test statistics and P values in medical papers
Emili García-Berthou and Carles Alcaraz
BMC Medical Research Methodology 2004, 4:13
To be published Friday 28 May 2004
Upon publication, this article will be available free of charge according to BMC Medical Research Methodology's Open Access policy at: http://www.biomedcentral.com/1471-2288/4/13
For further information about this research, please contact Dr Emili García-Berthou by telephone on 34-972-418-467 or by email at emili.garcia@udg.es
Alternatively, or for more information about the journal or Open Access publishing, contact Gemma Bradley by email at press@biomedcentral.com or by phone on 44-207-631-9931
BMC Medical Research Methodology (http://www.biomedcentral.com/bmcmedresmethodol) is published by BioMed Central (http://www.biomedcentral.com), an independent online publishing house committed to providing Open Access to peer-reviewed biological and medical research. This commitment is based on the view that immediate free access to research and the ability to freely archive and reuse published information is essential to the rapid and efficient communication of science. BioMed Central currently publishes over 100 journals across biology and medicine. In addition to open-access original research, BioMed Central also publishes reviews, commentaries and other non-original-research content. Depending on the policies of the individual journal, this content may be open access or provided only to subscribers.
Journal
BMC Medical Research Methodology