Series Comment – Biomedical research: increasing value, reducing waste By Professor Malcolm Macleod, Edinburgh University, Edinburgh, UK, et al
"Global biomedical and public health research involves billions of dollars and millions of people…Although this vast enterprise has led to substantial health improvements, many more gains are possible if the waste and inefficiency in the ways that biomedical research is chosen, designed, done, analysed, regulated, managed, disseminated, and reported can be addressed…In these papers, we set out some of the most pressing issues, recommend how to increase value and reduce waste in biomedical research, and propose metrics for stakeholders to monitor the implementation of these recommendations."
Series Comment – How should medical science change? By Richard Horton and Sabine Kleinert, The Lancet, London, UK
"On the basis of the evidence we present in this Series, a far broader question should be posed: how should the entire scientific enterprise change to produce reliable and accessible evidence that addresses the challenges faced by society and the individuals who make up those societies?"
Series 1 - How to increase value and reduce waste when research priorities are set
Sir Iain Chalmers, of the James Lind Initiative in Oxford, UK, and colleagues explain why decisions about what research to fund should be based on questions that are relevant to users of research and take more systematic account of relevant existing evidence.
The increase in annual global investment in biomedical research—reaching US$240 billion in 2010—has resulted in important health dividends for patients and the public. However, much research does not lead to worthwhile achievements, partly because some studies are done to improve understanding of basic mechanisms that might not have relevance for human health. Additionally, good research ideas often do not yield the anticipated results. As long as the way in which these ideas are prioritised for research is transparent and warranted, these disappointments should not be deemed wasteful; they are simply an inevitable feature of the way science works. However, some sources of waste cannot be justified. In this report, we discuss how avoidable waste can be considered when research priorities are set. We have four recommendations. First, ways to improve the yield from basic research should be investigated. Second, the transparency of processes by which funders prioritise important uncertainties should be increased, making clear how they take account of the needs of potential users of research. Third, investment in additional research should always be preceded by systematic assessment of existing evidence. Fourth, sources of information about research that is in progress should be strengthened and developed and used by researchers. Research funders have primary responsibility for reductions in waste resulting from decisions about what research to do.
Series 2 - Increasing value and reducing waste in research design, conduct, and analysis
Professor John Ioannidis, of Stanford Prevention Research Center in Stanford, USA, and colleagues consider improvements in the appropriateness of research design, methods, and analysis.
Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.
Series 3 - Increasing value and reducing waste in biomedical research regulation and management
Professor Rustam Al-Shahi Salman, of the University of Edinburgh in Edinburgh, UK, and colleagues address issues of efficient research regulation and management.
After identification of an important research question and selection of an appropriate study design, waste can arise from the regulation, governance, and management of biomedical research. Obtaining regulatory and governance approval has become increasingly burdensome and disproportionate to the conceivable risks to research participants. Regulation and governance involve interventions that are assumed to be justified in the interests of patients and the public, but they can actually compromise these interests. Inefficient management of the procedural conduct of research is wasteful, especially if it results in poor recruitment and retention of participants in well designed studies addressing important questions. These sources of waste can be minimised if the following four recommendations are addressed. First, regulators should use their influence to reduce other causes of waste and inefficiency in research. Second, regulators and policy makers should work with researchers, patients, and health professionals to streamline and harmonise the laws, regulations, guidelines, and processes that govern whether and how research can be done, and ensure that they are proportionate to the plausible risks associated with the research. Third, researchers and research managers should increase the efficiency of recruitment, retention, data monitoring, and data sharing in research through use of research designs known to reduce inefficiencies, and further research should be done to learn how efficiency can be increased. Finally, everyone, particularly those responsible for health-care systems, should promote integration of research into everyday clinical practice. Regulators and researchers should monitor adherence to each of these recommendations and publish metrics.
Series 4 - Increasing value and reducing waste: addressing inaccessible research
Dr An-Wen Chan, of the University of Toronto in Toronto, Canada, and colleagues examine the role of fully accessible research information.
The methods and results of health research are documented in study protocols, full study reports (detailing all analyses), journal reports, and participant-level datasets. However, protocols, full study reports, and participant level datasets are rarely available, and journal reports are available for only half of all studies and are plagued by selective reporting of methods and results. Furthermore, information provided in study protocols and reports varies in quality and is often incomplete. When full information about studies is inaccessible, billions of dollars in investment are wasted, bias is introduced, and research and care of patients are detrimentally affected. To help to improve this situation at a systemic level, three main actions are warranted. First, academic institutions and funders should reward investigators who fully disseminate their research protocols, reports, and participant-level datasets. Second, standards for the content of protocols and full study reports and for data sharing practices should be rigorously developed and adopted for all types of health research. Finally, journals, funders, sponsors, research ethics committees, regulators, and legislators should endorse and enforce policies supporting study registration and wide availability of journal reports, full study reports, and participant-level datasets.
Series 5 - Reducing waste from incomplete or unusable reports of biomedical research
Professor Paul Glasziou, of Bond University in Robina, Australia, and colleagues discuss the importance of unbiased and usable research reports.
Research publication can both communicate and miscommunicate. Unless research is adequately reported, the time and resources invested in the conduct of research is wasted. Reporting guidelines such as CONSORT, STARD, PRISMA, and ARRIVE aim to improve the quality of research reports, but all are much less adopted and adhered to than they should be. Adequate reports of research should clearly describe which questions were addressed and why, what was done, what was shown, and what the findings mean. However, substantial failures occur in each of these elements. For example, studies of published trial reports showed that the poor description of interventions meant that 40% were non-replicable; comparisons of protocols with publications showed that most studies had at least one primary outcome changed, introduced, or omitted; and investigators of new trials rarely set their findings in the context of a systematic review, and cited a very small and biased selection of previous relevant trials. Although best documented in reports of controlled trials, inadequate reporting occurs in all types of studies—animal and other preclinical studies, diagnostic studies, epidemiological studies, clinical prediction research, surveys, and qualitative studies. In this report, and in the Series more generally, we point to a waste at all stages in medical research. Although a more nuanced understanding of the complex systems involved in the conduct, writing, and publication of research is desirable, some immediate action can be taken to improve the reporting of research. Evidence for some recommendations is clear: change the current system of research rewards and regulations to encourage better and more complete reporting, and fund the development and maintenance of infrastructure to support better reporting, linkage, and archiving of all elements of research. However, the high amount of waste also warrants future investment in the monitoring of and research into reporting of research, and active implementation of the findings to ensure that research reports better address the needs of the range of research users.
Journal
The Lancet