CEBM and BMJ logo

What Colour is Evidence?

Dr Amy Price

Dr Amy Price

During the last decade a number of EBM proponents as well as critics have addressed the ‘problems with EBM’ with devaluing words and destructive proposals. The intentions of the EBM-black painters are not ours to justify. The most important results of these discussions remain an increased interest in EBM, deeper interest in the problems of the evidence base of modern medicine, the critical appraisal of research, and improving instruments for research methodology.

EBM-black painting serves as wolf-crying, it disorientates the public and brings intimidation, division and confusion to novices in EBM. This makes the progress of EBM more problematic when taken into the cultures and subspecialties where the critical appraisal of the evidence has yet to become known or accepted. Embracing error and the need to adapt by painting EBM all white is equally damaging as it denies the roots of critical appraisal. Constructive change will mean facing negative outcomes with a problem solving, solution orientated mindset that honors the roots of critical appraisal and the application of evidence which is at the core of EBM.

We suggest the alternative painting of EBM as a rainbow where evidence is celebrated and honored in the diversity needed to meet the needs of that individual patient who comes to us for help. This will provide room for the constructive change that is needed to grow in excellence, effectiveness and empathy. The rainbow was chosen because black absorbs all colors and darkens from within while the rainbow reflects the light that it finds in diversity and beauty.

This captures the essence of the abstract co-created by Professor Vasiliy V Vlassov and Dr Amy Price. The history of this article has evolved through the excellent contributions of the Evidence Based Health List Serve and some constructive thinking about what evidence means.

The Journey of Evidence Based Medicine | More Interesting Than a Paper

My idea was to capture this journey with PEE [Points | Examples | Evidence] write it up and discuss it in a simple paper but there were complications. The first glitch came in how the world defines the word Evidence. Professor Ben Djulbegovic and I explored this following a discussion on how differences in the understanding of common words can lead to working at cross purposes. Both parties hear the word but it is shaped by divergent world views and language interpretations.

How is the Word Evidence Understood?

We explored the multi-language meaning for the word evidence. The data showed two primary categories for evidence as translated into other languages: data, fact, testimony or observations, which per se do not make any link to the statement about the “truth”, and the inconvertible act of “proving” the “truth”3.

These common meanings are incompatible with how evidence is used and graded in EBM.2,3 The gaps in translation could expose EBM to probable assumptions and miscommunication. Our concepts of medical evidence and, thus, EBM are arguably shaped by translation. We contend that much of current criticism of EBM5 is rooted in misunderstanding the meaning of the concept of evidence.

The Call for Help to the EBH (Evidence Based Health) List serve

Dear all,

I am looking for concrete examples of EBM changing healthcare practice. I see arguments and methods but lack examples other than personal or with those I have worked with to say the practice of EBM changed healthcare specifically. E.G.  reduced harm be forcing the recall of a harmful drug, the discovery of a better way to treat a specific disease etc. Please help I think we need to build a history for the world outside EBM.

Best

Amy

And the EBH Responses Were:

“Success in children aematological tumours is a great example of what the scientific method can do.

Ambuj Kumar, Heloisa Soares, Robert Wells, Mike Clarke, Iztok Hozo,Archie Bleyer, Gregory Reaman, Iain Chalmers, and Benjamin Djulbegovic.

Are experimental treatments for cancer in children superior to established treatments? Observational study of randomised controlled trials by the Children’s Oncology Group. BMJ 331 (7528):1295, 2005 http://www.bmj.com/content/bmj/331/7528/1295.full.pdf .”

Contributed: Dr Federico Barbani

Do We Need Evidence to “Prove” EBM?

“I would suggest one does not need to have evidence that EBM changes healthcare – either positively or negatively.

In my mind, the concept of EBM or evidence-based healthcare (EBP) is a philosophy – it is an approach as to how we should practice heath care.

Personally, I believe one should for all health care decisions:

1) use the best available evidence

2) use one’s clinical expertise/experience, and

3) integrate patients’ values and preferences into the decision making process

Are there people who disagree with these three recommendations?

The reason I say it is a philosophy and not something that needs to show it improves outcomes is because using EBM could in theory “worsen” health outcomes.

As an example, let’s assume we did an RCT of statin use in primary prevention and randomized people to either taking a statin or EBP (making the choice for themselves)

For the sake of the argument let’s assume that:

1) statins definitely reduce the risk of cardiovascular disease by 25% or roughly 1-2% over 5 years in primary prevention

2) statins do NOT cause any side effects (I know this is not true)

3) let’s assume that most people when give the absolute benefit numbers would decide not to take a pill every day for five year

With these assumptions, the people that got randomized to choosing for themselves would have more CVD than the group that was “forced” to take a statin.

I would suggest that the outcome doesn’t matter – regardless of the outcome, practicing EBP is simply the right thing to do.”

Contributed: Dr James McCormack

Conceptual Agreement  

It may not be pure EBM but what about the Panorama programme that covered the discovery of Helicobacter Pylori

Contributed: Dr Neil Upton

What about any drug proved effective via an RCT?  Or systematic review?  Iain Chalmers often mentions streptokinase post-MI, but there are lots of other examples.

Contributed: Dr Jon Brassey

Is There Everyday Evidence for EBM?

“In workshops with GPs/family physicians, I point to research showing that the more you share evidence with patients, the less likely they are to want an intervention, such as warfarin for Atrial Fib or PSA screening for prostate cancer.

However, there is still some merit in looking for evidence that TRAINING practitioners to be Evidence Based (EB) produces Evidence Based practitioners. EB practitioners PRACTISING EB have better outcomes, defined otherwise than just event rates e.g. more cost-effective care post-stroke, more patient satisfaction.

In the UK, GPs’ performance in certain chronic conditions is monitored and financially rewarded. GPs who meet targets are  regarded in some circles as the high achievers whereas it could be argued, as I have done (http://bjgp.org/content/63/611/315.short), those who shares decisions with patients will appear to perform “badly”. 

Contributed: Dr Kev (Kevork) Hopayian

Teaching evidence based practice to improve patient outcomes

Emparanza, J. I., Cabello, J. B. & Burls, A. Does evidence-based practice improve patient outcomes. Journal of Evaluation in Clinical Practice, doi: 10.1111/jep.12460/full  Pre publication Access

Overturned by Evidence

“The WHI trial reversing the “standard of care” promoting postmenopausal hormone replacement therapy for cardiovascular disease prevention is one of the easiest to recognize, though data before the WHI trial was congruent — an EBM approach was different than the popular/accepted approach before the WHI trial was published.

The use of antiarrhythmic to treat PVCs after a heart attack (PVCs are associated with increased mortality and specific antiarrhythmic flecainide and encainide are effective in reducing PVCs) was a classic example in the need for a clinical outcome focus.  This was done as “good patient care” (Do you want to die while waiting for study results?) and a randomized placebo-controlled trial (the CAST trial) found the drug reduced PVCs but killed more patients.  This was the end of that practice”.

Contributed: Dr Brian S. Alper

“1. Smoking causes lung cancer. There was a time when we thought smoking was good for you until Doll and Hill’s studies showed otherwise. 

  1. Contaminated water and not air (miasma theory) spread Cholera by John Snow. Rightly named the “father of modern epidemiology”
  2. Studies showing babies sleeping on their stomach’s as recommended by the prominent pediatrician Benjamin Spock caused 1000’s of deaths due to SIDS. A great example of the fallacy of expert opinion. 
  3. I think it was a study which first questioned the blood-letting – a major treatment in medicine practiced for over 2000 years.
  4. Loftus studies showing the concept of “repressed memories” to be false”.

Contributed: Dr Anoop Balach   

RCTs Are Not the Lone Precursor to Major Change

The “trial” debunking blood-letting was an observational trial in about 1830 by Pierre Charles Alexandre Louis who found that patients with typhoid fever were more likely to die if they had their blood removed.  The likelihood that this result occurred by chance alone was less than one in ten thousand.

The Hill and Doll study was a case control study with an Odds Ratio that was very high (I cannot recall if it was 7 or 10, but either is very strong result). These both show that strong results can be obtained without doing expensive RCTs.  I believe that this is especially true when a very large result occurs.

Contributed: Dr Dan Mayer

Evidence Based Research for Midwives and Breastfeeding Mothers

A Cochrane review on continuity of midwife care was first published in 2004 and last updated in 2016. As more trials have been added to the Cochrane review uncertainties in the original findings have been reduced. Women who have received continuity of care from a midwife they know, rather than receiving medical-led or shared care, are:

  • 24% less likely to experience preterm birth,
  • 19% less likely to lose their baby before 24 weeks gestation, and
  • 16% less likely to lose their baby at any gestation.

These women are also more likely to have a vaginal birth, and fewer interventions during birth (instrumental birth, amniotomy, epidural and episiotomy), and are likely to have a more positive experience of labour and birth. These findings apply to both low- and mixed-risk populations of women, and there are no significant differences in outcomes between caseload and team care models.

This Cochrane review was identified as a priority review for updating by both the World Health Organisation and the Department of Health to inform the National Institute for Health and Care Excellence (NICE) review on the latest evidence on continuity of midwife care.

Evidence from the updated Cochrane review has had a significant influence on recent policy developments in relation to maternity care in both the UK and abroad.

  1. The results were cited as a key piece of evidence to inform models of care in Creating a Better Future Together – National Maternity Strategy 2016-2026, the first national maternity strategy for Ireland, which was published in January 2016.
  2. Evidence from the review was also cited in the National Maternity Review for England published in February 2016 (Better births – Improving outcomes of maternity services in England), led by Baroness Cumberlege and conducted as part of the NHS England Five Year Forward View.
  3. They have also informed the RCM/RCOG statement on continuity of carer and multi-disciplinary working published in April 2016.
  4. Internationally, the Cochrane review was cited in a Lancet series on midwifery which aimed to inform workforce and health system development plans under the United Nations’ Post-2015 Development Agenda.

Sandall J, Soltani H, Gates S, Shennan A, Devane D (2016). ‘Midwife-led continuity models versus other models of care for childbearing women‘. Cochrane Database of Systematic Reviews 2016, Issue 4. Art. No.: CD004667.

Contributed: Dr Jane Sandall

These three evidence based reviews have been the most significant in early postnatal care/ infant feeding. The timing of the introduction for solid food, kangaroo care for low birth weight babies and skin to skin for term infants.

Contributed: Phyll Buchannon

Thoughts for the Future

We all benefit from promoting the discovery of better treatments as we grow in communities of practice. Through the practice of EBM it is possible to speed up the acceptance of the best treatments available and to contribute to the elimination of useless/dangerous interventions.

EBM may indirectly influence the FDA, regulators, and consumers, who in turn influence providers. Let us work to build bridges of Evidence Based Medicine with all stakeholders as building better healthcare is the challenge of a lifetime and every person represents a life that matters.  

One limitation EBM and in fact any organization faces is that evidence and knowledge translation are dynamic. The challenge is we can synthetize the evidence we have available today but tomorrow may bring new discovery or uncover hidden evidence. Answers tend to reflect the knowledge collected, it is bounded by that we think we know and this is not always the objective or full faceted truth. Facing the challenges that require change constructively and growing with grace will continue to build EBM from the inside out.

2017 Highlights