Better Decisions Require Better Evidence: Part 1

The campaign starts at EvidenceLive 2016 – with  an open meeting to prioritise and explore the potential solutions to better evidence for  better decisions.

Carl Heneghan

At the core of evidence-based medicine is the integration of patient values and high quality evidence. Informed patients should also understand their treatment options and actively participate in making decisions about their own health, and  to achieve this, clinicians require better research evidence. However, there are growing concerns that a sizeable amount of current published research is irrelevant, wasteful and detrimental to patient care.

In 1994, Doug Altman, pointed out in an editorial on the scandal of poor medical research we need less research but better research. Has anything therefore changed – for the better – in the last 20 years? Searching PuBMed Trend, which displays the number of articles in  Medline by year, reveals 9,613 trials were published in the same year as the Altman editorial.  Twenty years later this number has risen, more than threefold, to 31,648 trials. So much for less research.

Whilst trials have been accruing we have not observed better research. Instead, significant increases in poor quality practices have occurred: John Yudkin and colleagues argue that our ‘obsession with surrogates is damaging patient care:’ they are easier to measure are increasingly used instead of important patient outcomes  and are on the rise. A systematic review by Cordoba and colleagues concluded that composites are often unreasonably combined, poorly defined and reported, and often leading to exaggerated outcomes. An analysis by Jamie Kirkham and colleagues of 283 cochrane reviews revealed that outcome reporting bias is an under-recognised problem and affects over half of cochrane reviews and often affects outcomes. Eve Tang and colleagues, analysis of  300 trials, with serious adverse events reported on ClinicalTrials.gov. found a quarter did not have a corresponding publication, and an analysis by Rodgers and colleagues of confidential and published data on rhBMP-2 for spinal fusion concluded the  “reporting of adverse event data in trial publications was inadequate and inconsistent to the extent that any systematic review based solely on the publicly available data would not be able to properly evaluate the safety.”

Outcome switching: The Compare Projects analysis of outcome switching in clinical trials of the top five medical journals worryingly reports that on average each trial only reports 62% of its specified outcomes. Finally, Andreas Lundh and colleagues analysis of industry sponsorship concludes that an “industry bias exists that cannot be explained by standard ‘Risk of bias’ assessments.

Poor quality research evidence is now so commonplace it is imperceptibly ingrained in the culture. As an example, in a report to David Sackett, John Ionnadis considers evidence-based medicine has been hijacked, and as a consequence, drug trials are now mainly done by industry for their benefit.  

In a recent BMJ editorial on fixing the problems in medical research  Goldacre and myself  called  for independent trials, reducing research costs, incentivising better evidence, declaring conflicts of interest and a thorough transformation in the public understanding of evidence to support healthcare decision making.   

There is little wrong with the fundamental principles of EBM. But for too long we have tolerated an evidence-base distorted by systematic bias and conflicts of interest. If ignored, the inherent problems in current research production and publication may become entrenched and insolvable, that is, if some of the problems aren’t already. What we now need is firm action to develop better evidence to support better decision making.

The next in this series will look at better decisions require more affordable treatments.    

Carl Heneghan is professor of EBM at the University of Oxford, director of CEBM, which co-organizes the EvidenceLive conference with the BMJ

His research interests span chronic diseases, diagnostics, use of new technologies, improving the quality of research and  investigative work with The BMJ on drugs and devices that you might stumble across in the media.  

I declare that I have read and understood BMJ Policy on declaration of interests and I hereby declare the following interests:  CEBM jointly runs the EvidenceLive conference with The BMJ and is one of the founders of the AllTrials campaign. He has received expenses and payments for his media work  and has received expenses from the World Health Organization (WHO) and and holds grant funding from the NIHR, the National School of Primary Care Research, the Wellcome Trust, and the WHO.