Author Archives: Carl Heneghan

About Carl Heneghan

Carl is Professor of EBM & Director of CEBM at the University of Oxford. He is also a GP and tweets @carlheneghan. He has an active interest in discovering the truth behind health research findings

Evidence Live 2018

On behalf of the Evidence Live Programme Committee thank you for joining us at our 7th Evidence Live conference in Oxford last month. During the planning stages we very much consider what the output will be for our delegates and hope that we meet our objectives.  We were fortunate to have a wide range of high profile speakers from across the EBM population and associated organisations, content, quality of the topics covered and inclusivity all received positive feedback during the event.  All further comments are gratefully received, please take a minute (or five) to complete the FEEDBACK QUESTIONNAIRE so that we can keep on improving on the experience.

Evidence Live 2018 OPEN DELEGATE LIST

POSTER ABSTRACTS Evidence Live 2018

ORAL ABSTRACTS

BMJ Evidence-Based Medicine Journal Evidence Live Conference Supplement

2018 Conference Programme – Keynote and Workshop Abstracts

PRESENTATIONS

Ways of making evidence used and useful in a world of information overload – Tara Lamont

Manifesto Update – Carl Heneghan

From 10 essentials for EBM to 10 essential papers on bias – David Nunan    Catalogue of Bias – David Nunan & Carl Heneghan

Research Integrity QT – Carl Heneghan  Research Integrity QT – Emily Sena  Research Integrity QT – Peter Wilmshurst

The false positive risk: a proposal concerning what to do about p-values – David Colquhoun

Dissemination and application of research results to inform decision making – Peter Davidson  Summary – Davidson Evidence for decision-makers

Key concepts for teaching critical thinking and critical appraisal – Oxman, Chalmers, Al Barqouni, Carter

Building capability, leadership and a home for Evidence-based Medicine in Ireland – Niamh ORourke

Informing, Developing & Implementing

Evidence Live program for 2018: Informing, developing and Implementing evidence for real change 

The Evidence Live’s program for 2018 contains one of the best line-ups in the seven  years the conference has been informing, developing and implementing evidence for real change.

It’s so good; it’s hard to know where to start.

Well, let’s start with day one: Tara Lamont is Deputy Director NIHR DC  and has worked for over twenty years in health services research, audit and patient safety.  She will be talking about ways of making evidence used and useful in a world of information overload. On the same day, Research Integrity Question Time will cover integrity in clinical, academic and animal research.  The audience will hear from an advocate (check out the line-up here) in each area setting the scene and generating questions for the bigger debate to come.

On day two we’ll be hearing about Science & Informing the Public with

Margaret McCartney, Andy Oxman & Tracey Brown. Why is the public still being misled asks Margaret McCartney?  Andy Oxman will likely tell us it’s because we aren’t teaching children sufficiently to detect such misleading claims. This session will be followed by a panel session from some of the best healthcare journalists in Europe to find the truth through medical journalism: Shelley Jofre, Jet Schouten, Kath Sansom & Deb Cohen

On the final day, we’ll be hearing about communicating Healthcare Findings to Inform All from  Ben Goldacre, Peter Davidson, Lisa Schwartz & Steve Woloshin. Finally, we will be closing with the dissemination of Cochrane Reviews: maximising research Impact to improve healthcare by  David Tovey, Editor in Chief Cochrane Library chaired by  Fiona Godlee

In between, there will be workshops, parallel sessions, posters, breakfast sessions and an update on the EBM manifesto.  See you there.

Carl Heneghan, Director CEBM

Evidence Live 2018 Symposium

The Evidence Live 2018 Symposium. June 18-20

A 3-day conference jointly hosted by the Centre for Evidence-Based Medicine at the University of Oxford and The BMJ.

Square Peg, Round Hole

As the seventh Evidence Live conference approaches I find my time is taken up with thoughts of  ‘which room should be allocate for which talk, how to display a landscape poster on a portrait poster board (the boards are portrait), is there enough catering and what is the coffee runs out’.  The curse of an internal conference organiser is that of getting too involved in the detail and overlooking the content.   Every year I tell myself  to be more immersed and this year I actually am.
So what will be will be, abstracts are accepted, rooms are booked and the procesco is on ice. Lets take a look at the programme.

This year we will focus on how dissemination of evidence can be radically improved; how evidence can be translated more efficiently into practice; how innovation with high quality evidence can drive substantial change.

Global leading speakers who actively disseminate and translate evidence to drive innovation and a better public understanding health research will inform and inspire you to continue improving local and global healthcare systems. Focusing on the three main pillars of dissemination, translation and innovation Evidence Live 2018 aims to create unique initiatives that will enhance your ability to communicate, translate and exchange information to make a real difference.

Tara Lamont – Deputy Director NIHR Dissemination Centre opens with  – Ways of making evidence used and useful in a world of information overload

New for 2018 featuring Aseem Malhotra, Emily Sena, Peter Wilmshurst & Trish Groves
15:45 Monday June 18th Question Time where serious questions on integrity across Clinical, Animal and Academic research will be discussed.

DRINKS RECEPTION – Radcliffe Observatory Quarter.  Networking opportunity.

Margaret McCartney100% safe, 100% bull – why are the public still being misled? is joined by Andy Oxman & Tracey Brown in a session focused on Science & Informing the Public followed later with a session from journalists/patients Shelley Jofre (BBC), Jet Schouten (AVROTROS), Kath Sansom (Independant) & Deb Cohen (BMJ) who will give an insight into Finding the Truth through Medical Journalism.

CONFERENCE DINNER – Somerville College, University of Oxford.  More opportunity to network and start potential collaborations.

On Wednesday morning our very own Ben Goldacre, defender of the faith is billed alongside Peter Davidson Director of the NIHR Dissemination Centre and  Lisa Schwartz & Steve Woloshin from the Dartmouth Institute for Communicating Healthcare Findings to Inform All.  A session that will cover the most effective ways to exchange information across difference receiver groups.

Closing with Dissemination of Cochrane Reviews. David Tovey in conversation with Fiona Godlee, addressing the main objective Maximising research Impact to improve healthcare

There is so much more than this across the 2.5 days.  Please take a look at the programme for details on workshops and seminars also taking place. Parallel session talk titles will be available soon.  Registration OPEN with day rate options available. Hope to see you in June.

Ruth

Stimulating, Provoking, Entertaining & Inspiring

 

DEADLINE  February 28th
Submissions are being accepted under the following themes

  • Educating patients, the public and healthcare providers in making informed choices
  • Improving the dissemination of evidence
  • Making research evidence relevant, replicable and accessible
  • Better, usable and more accessible clinical guidelines.
  • Better use of real world data
  • Building Capability and Leadership
  • Increase the systematic use of existing evidence
  • Improve the use of information in the media
  • Translation of evidence into better care
  • Tools and concepts that are basic and central to the teaching and practising of evidence-based medicine

 

Finding the Truth through Medical Journalism

Evidence Live 2018 Medical Journalism session Tuesday June 19th will explore the benefits to health through the media.

Harold Evans was a UK journalist, who grew up round the corner from me in Newton Heath, Manchester. Evans became the editor of the Sunday Times in 1967, and for fourteen years sought to bring to the public attention scandals that were being ignored.  One of these was thalidomide. The Times’s investigative team (called Insight) fought for higher compensation for victims and also revealed the manufacturers had not done the necessary level of testing required at the time before distributing.

Journalism, therefore, can act as a force for public good, and when it comes to health can work in the public interest to ensure that those stories that are ignored, denied or defended are told.

At Evidence Live 2018  we’ll be exploring the role of journalism in maintaining public health, upholding the trust and making for better healthcare.

Jet Schouten – ‎Investigative Journalist from RADAR TV, a Dutch consumer television, programme, went undercover to expose how the regulation of medical devices is so lax that mesh packaging for fruit could be approved as a medical device to be implanted in people’s bodies.

Kath Sansom – Journalist & founder of Sling the Mesh – has campaigned to ensure women injured by pelvic mesh implants have told their stories. She said to Sky News: ‘It probably needed a journalist to be mesh injured to provide the final media push needed for the issue to get to Westminster.’

Shelley Jofre – BBC Panorama. Her first Panorama programme was an investigation into the over-prescription of psychiatric drugs to children, and since then she has investigated prescription tranquilizers and non-addictive anti-depressants like Seroxat impacting on drug’s safety and the reporting of side effects.

Deborah Cohen – is investigations editor of the BMJ and much of her work has focused on drug and medical device regulation, access to clinical data and conflicts of interest. She has collaborated on documentaries with the BBC, Channel 4 News and Dispatches, Al-Jazeera, as well as having worked with the Cochrane Collaboration and the Bureau of Investigative Journalism.

Carl Heneghan Feb 20th 2018

Finding Solutions to the Lack of Integrity in Research

Evidence Live 2018 Research and Integrity session Monday June 18th will discuss the answers to the growing lack of just that!
Carl Heneghan sets out some of the important issues.

What defines a problem is a difference between what’s happening and what you would like to be happening. For the research integrity problem, I can tell you a lot about what is happening: competing interests in research, increased global competition and academic pressures are undermining the integrity of science. [1, 2]

A small proportion of scientists admit to the fabricating, falsifying or modifying data or results. But an awful lot more regularly acknowledge questionable research practices that include data mining for statistically significant effects, selective reporting of outcomes, switching outcomes, publication bias, protocol deviations and concealing conflicts of interest.[3] A 2012 survey of 9,036 BMJ authors and reviewers showed that of the 2,782 (31%) who replied, 13% had witnessed or had first-hand knowledge of UK based scientists or doctors inappropriately adjusting, altering or fabricating data during their research or for publication. [4]

Such financial and non-financial conflicts of interests are a widespread phenomenon amongst academics and academic institutions. Conflicted researchers are common (one in four investigators have been found to have industry ties) and associate with pro-industry conclusions, withholding of results and lack of data sharing. They also overtly influence guidelines and latterly have infiltrated patient advocacy organisations. [5, 6]

What are the solutions to deter questionable research practice and promote research integrity? David Goodstein review tells us that three motives are ever present when it comes to fraud: individuals were a) under career pressure; b) thought they knew what the result would be if they went to all the trouble of doing the work correctly; and c) was in a field in which studies are not expected to be precisely reproducible.  [7, 8]

Also, the “publish or perish,” model of research creates career pressure and institutional pressure that promotes shortcuts and sloppy, flawed research. The current system is increasingly contributing to this problem. [9]

What are the solutions to deter questionable research practice and promote research integrity? My top three measures to mitigate questionable research practices should consider: a central open-access database for all financial and non-financial conflicts of interest; a system to address the perverse research incentives that act as barriers to reproducibility and replicability and strengthen of the post-marketing surveillance of drugs to solve the problem of underreporting of harms

I’m looking to the debate at Evidence Live to provide more practical measures that can drive integrity in research and promote the production of high quality evidence that matters to patient care.

Carl Heneghan is Professor of EBM at the University of Oxford, Director of CEBM and Editor in Chief of BMJ EBM

If you want to join the mandate to prioritise and carry out the solutions to the deluge of poor quality evidence and promote research integrity  then why not join us at Evidence Live 2018?

References

  1. Alliance, T., Treatment Action Group Issues Global Tuberculosis Research & Development Update. 2010.
  2. Martinson, B.C., M.S. Anderson, and R. de Vries, Scientists behaving badly. Nature, 2005. 435(7043): p. 737-8.
  3. Nippert, I., B. Edler, and C. Schmidt-Herterich, 40 years later: the health related quality of life of women affected by thalidomide. Community Genet, 2002. 5(4): p. 209-16.

4 ICJME, Uniform Requirements for Manuscripts Submitted to Biomedical Journals:

  1. Norris, S.L., et al., Conflict of interest in clinical practice guideline development: a systematic review. PLoS One, 2011. 6(10): p. e25153.
  2. Lundh, A., et al., Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue – cohort study. PLoS Med, 2010. 7(10): p. E1000354.
  3. Ghostwriting Revisited: New Perspectives but Few Solutions in Sight. PLoS Med, 2011. 8(8).
  4. SINGER, D.W.a.N., Ghostwriting Is Called Rife in Medical Journals New York Times 2009. Accessed at: http://www.nytimes.com/2009/09/11/business/11ghost.html.
  5. Publish or Perish: What are its Consequences? – Enago Academy. Enago Academy. 2015. https://www.enago.com/academy/publish-or-perish-consequences/ (accessed 8 Mar2017).

Evidence-Based Medicine Manifesto – an update

We are working to identify and prioritise those tasks likely to improve the quality of evidence used at the bedside and make for better healthcare.
Carl Heneghan

In 2017 we published the Evidence-Based Manifesto  as a “response to systematic bias, wastage, error, and fraud in research underpinning patient care.”

The BMJ and the Centre for Evidence-Based Medicine have collaborated on Evidence Live, since 2010. One of the central themes that have emerged throughout has been the substantial problems that exist with the quality of the evidence, the potential solutions that have been tried and their subsequent impact on practising EBM on the ground.

At Evidence Live 2017, we presented the manifesto commitments. Jeff Aronson, in the ‘A Word About Evidence’ series on Manifesto, tells us that “during elections, parties may issue manifestos in which they outline their plans, which, if they are elected, they have a mandate to carry out. If they do not do so, or if they do things the manifestos did not include, they can be accused of abusing their mandate.”

 

We are therefore working to carry out the manifesto commitments – with your help.

At the 2017 conference, we did a workshop where we said we would identify, take forward and implement solutions for better evidence in respective fields. This workshop, attended by over 200 participants, facilitated the generation of a jobs list and identified potential solutions across different stakeholder groups. We also solicited ideas through a form posted on the Evidence Live site.

Overall, we got 140 unique responses. Turns out to be the largest focus group I’ve participated in; the number of responses means it’s taking me some time to analyse the themes.

The focus group work identified sixteen different stakeholder groups that could undertake tasks related to the EBM Manifesto. The most referred to groups were journal editors, funders, academic researchers, clinicians, patients – the ones you’d expect. But there were also some you might not: Research Excellence Framework in the UK Equator network, medical writing companies, ethics committees, head of the foundation program for training, guideline developers, professional bodies, those who set the curriculum for health professionals and academic institutions.

You could refer to these as the EBM Manifesto ecosystem that we want to target.

What should we do next? I’d be interested in your thoughts. Should we appoint a couple of people to be theme leads for each stakeholder group journal eds, funders etc., and then seek input to prioritise the tasks at hand, mobilise forces and set to work. We should be able to prioritise those tasks that are likely to improve quality of evidence used at the bedside and make for better healthcare.

I look forward to the ongoing conversation at Evidence Live 2018 where we will present the jobs list for the EBM manifesto ecosystem.

Carl Heneghan is Professor of EBM at the University of Oxford, Director of CEBM and Editor in Chief of BMJ EBM


If you want to join the mandate to prioritise and carry out the solutions to the deluge of poor quality evidence then why not join us at Evidence Live 2018?

 EBM manifesto for better healthcare

► Expand the role of patients, health Professionals and policy makers in research.

►Increase the systematic use of existing evidence.

►Make research evidence relevant, replicable and accessible to end users.

►Reduce questionable research practices, bias and conflicts of interests.

►Ensure drug and device regulation is robust, transparent and independent.

►Produce better usable clinical guidelines.

►Support innovation, quality improvement and safety through the better use of real-world data.

►Educate professionals, policy makers and the public in evidence-based healthcare to make an informed choice.

►Encourage the next generation of leaders in evidence-based medicine

Heneghan C, Mahtani KR, Goldacre B, Godlee F, MacDonald M, Jarvies D Evidence-based medicine manifesto for better healthcare. BMJ 2017;357:j2973.

Competing interests

Carl has received expenses and fees for his media work. He holds grant funding from the NIHR, the NIHR School of Primary Care Research, The Wellcome Trust and the WHO. CEBM jointly runs the EvidenceLive Conference with the BMJ and the Overdiagnosis Conference with international partners which are both based on a  non-profit model.

 

 

A Word About Evidence: 3. Manifesto

A manifesto for Evidence-Based Medicine (EBM) was published in the BMJ earlier this year and presented atEvidence Live.

Jeff Aronson has been thinking again about the word manifesto.


The Indo-European root MAN meant a hand. The Latin word was manus, from which we get words such as maintain, manacle, manage, manège, manicure, manipulate, manner, manoeuvre, mansuetude, manual, manufacture, [manu]factory, manumit, and manuscript. Manqué, an adjective that refers to an unfulfilled ambition (e.g. a scientist manqué), is from the Latin mancus, having a useless hand. To emancipate is to take someone by the hand (Latin capere, to take). An amanuensis (short for servus a manu plus the suffix -ensis, belonging to) writes things out by hand at another’s dictation, as Tolstoy’s wife, Sofya Andreevna, did for him and as Eric Fenby did for the composer Frederick Delius. The manubrium sterni is like a handle, comparing the body of the sternum (Greek στέρνον the chest) to the blade of a sword, and the xiphoid process (ξίϕος a sword) to its tip.

Add Latin manus to [in]festus, hostile or troublesome, and you get manifestus, caught in the act, red-handed, and therefore obvious, manifest. As Dryden wrote in Absalom and Achitophel, “Now, manifest of Crimes, contriv’d long since, He stood a bold Defiance with his Prince”. A manifest thus became a piece of evidence, such as the list of a ship’s cargo or any inventory, and a manifesto, from the Italian form of the word, then became evidence of intention: “a printed declaration, explanation, or justification of policy issued by a head of state, government, or political party or candidate.”

The earliest reference in English to a manifesto of this sort is to be found in Nathaniel Brent’s translation of Paolo Sarpi’s History of the Council of Trent (1620). Theodore Alois Buckley’s history of 1852 makes it clear that manifestos were commonly compiled at the time of the Council, which was an ecumenical council of the Roman Catholic Church held between 1545 and 1563. When, for example, the Emperor Charles V declared war on the Protestants in 1546, the elector of Saxony and the Landgrave of Hesse published a manifesto, declaring that “the war was really undertaken in the cause of religion”. This laid at least some of the blame on the Council and the then Pope, Paul III (Alexander Farnese), with whom the Emperor had a treaty, although the Emperor had declared it to be a secular campaign, not wanting his wishes to be associated with those of the Pope.

A manifesto is also “a book or other work by a private individual supporting a cause”, like Atul Gawande’s Checklist Manifesto (2009) and my 2010 manifesto for UK clinical pharmacology. The 1848 political pamphlet Manifest der Kommunistischen Partei (The Communist Manifesto; picture), by Karl Marx and Friedrich Engels, is perhaps the most famous example. Tsar Nicholas II issued the October Manifesto in 1905, promising political reforms and aborting the first Russian Revolution. Artists are fond of publishing manifestos describing their creeds; well-known examples include the Russian Futurists’ ManifestoA Slap in the Face of Public Taste (1912), and the Surrealists’ Manifestos (1924) by André Breton. There are many other examples of varying seriousness.

Searching PubMed for the term “manifesto” in the titles of papers, I found nearly 250. The earliest all dealt with alcohol. The first, published anonymously in the British Medical Journal in 1902, was titled “A medical temperance manifesto” and reported that the Council of the British Medical Temperance Association in conjunction with the American Medical Temperance Association and the Association of German-speaking Medical Men, had agreed to issue an international manifesto on abstinence from alcoholic beverages. Their manifesto was again advertised in the journal in 1903 and later that year in the California State Medical Journal. The database lists no further manifestos until 1935, when the subject was again alcohol, again in the British Medical Journal, “The Brewers’ Attack upon Youth”, advertised in an article titled “The Beer Drinking Habit: Doctors’ Manifesto”; signed by “a thousand medical men”. This manifesto inveighed against a campaign by the Brewers’ Society “to get the Beer Drinking Habit instilled into Thousands, almost Millions, of Young Men who do not at present Know the Taste of Beer” (punctuation in the original). The 2012 legislation in Scotland on minimum unit pricing, delayed because of challenges from the Scottish Whisky Association, but now to be implemented from May 2018, could be considered the latest in this series of manifestos.This puts pressure on the rest of the UK to do likewise. The Welsh Parliament is already considering legislation.

Since 1935, papers about manifestos have appeared sporadically. PubMed lists only 25 in the 84 years from 1902 to 1985. Since then the numbers have been variable (Figure 1), with a marked increase since 2010, although the numbers are still small. The main topics include public health, women’s health, the UK’s National Health Service, and various aspects of research and policy, including evidence-based medicine.

Figure 1. The numbers of papers containing the word “manifesto” in titles since 1986 (source PubMed); the 15 papers published in 1997 included a series of 12 titled “Osteopathic manifesto.”

The purpose of the EBM manifesto is to develop, disseminate, and implement better evidence for better healthcare in response to systematic bias, wastage, error, and fraud in research underpinning patient care. Its nine principles are to:

  • expand the roles of patients, health professionals, and policymakers in research;
  • increase the systematic use of existing evidence;
  • make research evidence relevant, replicable, and accessible to end users;
  • reduce questionable research practices, bias, and conflicts of interests;
  • ensure that drug and device regulation is robust, transparent, and independent;
  • produce better usable clinical guidelines;
  • support innovation, quality improvement, and safety through the better use of real-world data;
  • educate professionals, policymakers, and the public in evidence-based healthcare, so that they can make informed choices;
  • encourage the next generation of leaders in evidence-based medicine.

During elections candidates or their parties may issue manifestos in which they outline their plans, which, if they are elected, they have a mandate to carry out. If they do not do so, or if they do things the manifestos did not include, they can be accused of abusing their mandate. Few people, I suspect, read political manifestos, which often contain weasel words and vague, platitudinous promises. It would be good to see a political manifesto that contained evidence instead.


Jeffrey Aronson is Associate Editor BMJ EBM, consultant physician and clinical pharmacologist, and Fellow of CEBM
Conflict of Interest: none declared
Originally published on BMJ EBM Spotlight January 9th 2018 – http://blogs.bmj.com/bmjebmspotlight/2018/01/09/word-evidence-3-manifesto/

What I learned from Evidence Live 2017: Fixing the “E” in EBM

On the longest (and probably hottest) day of 2017, Evidence Live kicked off at the Blavatnik School of Government, University of Oxford. Professor Carl Heneghan, new Editor-in-Chief of BMJ Evidence-Based Medicine (EBM), opened the conference with the EBM manifesto which recognizes problems in research evidence and proposes actions needed to tackle these problems. This is my summary on what I learned about fixing the “E” in EBM during Evidence Live.

What is wrong with evidence? The scandal of bad medical research

In his 1994 paper, “The scandal of poor medical research”, Doug Altman argued we needed “less research, better research, and research done for the right reasons.” Despite some progress (e.g. CONSORT statement), medical research continues to suffer from bad conduct and reporting. For example,

  • Ben Goldacre (University of Oxford): Checking five major medical journals (COMPare Trials) and ClinicalTrials.gov (Trials Tracker) showed outcome switching is common in clinical trials and many pharmaceutical companies, government agencies, and universities do not report all trial results.
  • Mary Dixon-Woods (University of Cambridge): “Verschlimmbesserung: an attempted improvement that made things worse than they already were.” Although this is inevitable in quality improvement projects, methods for evaluating quality improvement interventions often lack scientific rigour to overcome the underlying bias favouring improvement.
  • Jong-Wook Ban*: A mixed methods study showed cardiovascular clinical prediction rules are often developed without providing clear justification by citing existing rule, although most authors agree citing existing rule is important.
  • Jamilla Hussain (Hull York Medical School): A systematic review of clinical trials in palliative care showed information about missing data was often incompletely reported and mishandling of missing data was common.
  • Tone Westergren (Oslo University Hospital): An analysis of randomized clinical trials included in a systematic review showed few reported adverse effects as recommended in the CONSORT harms extension.

What solutions were proposed?

Many innovative ideas addressing these problems were exchanged at Evidence Live 2017. Some of the proposed solutions are outlined below:

  • Expanding the role of patients in research: Simon Denegri (NIHR, INVOLVE) discussed how involving patients and the public in design and delivery of research might improve relevance and efficiency. Amy Price (University of Oxford) described how to report patient and public involvement in research.
  • Increasing the systematic use of existing evidence: Jon Brassey (TRIP) proposed a community rapid review system where users conduct and update rapid reviews with support from community and technology.
  • Producing better usable clinical guidelines: Eve O’Toole (National Cancer Control Programme) shared methods for developing clinical guidelines using a rapid review process that limits the number of questions by focusing on areas with new evidence, variation in practice, and potential for clinical impact.
  • Making research evidence relevant and accessible to end users: Clinicians often fail to use research evidence because they lack skills to find, appraise, and apply relevant evidence in practice. Caroline Blaine (BMJ Knowledge Centre) showed how this challenge may be addressed by using point-of-care tools such as BMJ Best Practice. Additionally, shared decision making may be facilitated by using decision aids in anticoagulation care (Peter Oettgen, DynaMed Plus) and for women with heavy menstrual bleeding (Rachel Thompson, Dartmouth Institute).
  • Reducing questionable research practices: Guidelines aimed to improve reporting were proposed for surgical case reports (SCARE statement), surgical case series (PROCESS statement), and adaptive clinical trials (ACE project).
  • Ensuring drug regulation is transparent and independent: To maintain public trust in medicine, Fergal O’Regan (the European Ombudsman) stressed that keeping transparency in the drug approval process, including pre-authorization and post-authorization evaluation, is essential.
  • Using real world data to support innovation, quality improvement, and safety: Routinely collected data can be used to complement the results of randomized controlled trials (Lars Hemkens, University Hospital Basel), develop prediction models (Katriina Heikkila, London School of Hygiene and Tropical Medicine), prioritize research agendas (Fay Chinnery, Wessex Institute), monitor evidence-based practice (Jamie Falk, University of Manitoba) (Kelsey Chalmers, University of Sydney), and assess burden of disease (Gloria Ansa, University of Ghana).
  • Educating professionals and the public in EBM to make an informed choice: Sharon Mickan (Griffith University) showed small group education improved confidence levels and positive behaviours in the use of evidence-based practice among clinicians. Matt Oxman (Norwegian Institute of Public Health) presented a cluster randomized controlled trial that showed providing the Informed Health Choices resources lead to better assessment of claims about treatment effects by primary school children.

So, what would you do to help fix the “E” in EBM? Let’s bring your ideas, experiences, and challenges back to Evidence Live 2018. Until then…

* Jong-Wook Ban, MD, MSc

I am a General Internist in Olympia, WA and a DPhil candidate in Evidence-Based Health Care with interests in research inefficiencies in cardiovascular prediction rule development.

Reflection on Attending Evidence Live 2017: The Era of Cross-border Collaboration Has Come!

Junqiao Chen

From June 21 to 22 of 2017, I attended the Evidence Live conference at University of Oxford. On reflection, one theme emerges within the diverse topics presented in the conference, that is the era of cross-border collaboration has arrived. By ‘border’, I mean both professional borders, organizational borders, and geographical borders.

The old time of working in silo is ending. Many speakers in this conference pointed out that in this old paradigm, many funders set their own agenda without wider input. Many investigators conduct trials without learning from prior studies, resulting in the sad fact that a large proportion of studies have serious methodological limitations, or the ‘Scandal of poor medical research’ described by speaker Doug Altman. After completion, results are not reported (in full), and funders don’t audit this waste of money. For those getting published in peer-reviewed journals, many still suffer from poor quality. The bi-directional link between the publication of primary studies and systematic reviews is broken. Primary studies may have added more noise than signal into the evidence base, and when there is a true signal, systematic reviews are not able to follow accordingly. Even when the evidence is accurate and updated, many local quality improvement activities have failed. Lastly, patients are not evolved in these steps.

In the new emerging paradigm, first and foremost, patients and the public are involved to help improve health care research and delivery, as pointed out by speakers Simon Denegri, Trish Groves, and James Munro. I am transformed when hearing the work done by James Lind Alliance on involving patients to prioritize research agenda. On reflection of my own research, I feel a little bit shamed that I never engaged patient in the process, thus I am not confident at all that my research is relevant to them. I also get super excited after learning about Care Opinion. Traditional evidence-based medicine tells us what works. New experience-based medicine tells us what matters. It is a trusted resource for clinicians to learn from patients.

Second, trials could be done in a more collaborative and creative way. Speaker Lars Hemkens gave an example of doing a trial in Switzerland where there was no need to have a dedicated research nurse to collect data on one primary outcome – 30-days mortality. This is because it is a universal practice in Switzerland to follow up with patients after 30-day of discharge. If we could incorporate this type of routinely collected data, the conduct of trials could become more efficient.

Third, after the research is completed, funders and journals need to enforce transparency in reporting, as pointed out by speaker Iain Chalmers and Doug Altman in their talk on REWARD Alliance and EQUATOR network. When we find breach in research integrity, we need to have an open feedback loop from peers, as suggested by Ben Glodacre in his talk about audit and accountability of research quality.

When a primary study is published, authors need to inform systematic reviewers of relevant topic to update the review (when necessary), as audience member and presenter Rabia Bashir came to a consensus after she presented her finding that systematic reviews are not being updated in areas where evidence is accumulating. This is easier said than done (as mentioned earlier, many primary studies were performed without knowledge of an existing systematic review). However, we are not disparate because it is becoming easier for systematic reviewers to identify randomized controlled trials now, as presenter Anna Noel-Storr demonstrated the incredible success of Cochrane Crowd, which allows tens of thousands of volunteers around the world to contribute to the classification of articles. Crowdsourcing is not only happening on the level of article screening, as speaker Jon Brassey is developing a “community rapid review” functionality in the TRIP database (an EBM-enriched search engine), which will allow user of TRIP to construct a rapid review after using TRIP to solve their clinical questions. No matter where you live and what you do, we all benefit from faster and more accurate review of evidence. A quick comment on this subject is that it is still not very clear when rapid review might work and might not work. I proposed a pilot-tested study design in my presentation and called for wider participation.

However, even we have the best evidence about what seems to improve quality of care, it only describes a very small portion of what might happen in the real world and it is hard to reproduce. As speaker Mary Dixon-Wood pointed out in an earlier paper that ‘the superficial outer appearance of the intervention or QI [Quality Improvement] method is reproduced, but not the internal mechanisms (or set of mechanisms) that produced the outcomes in the first instance’ (Dixon-Woods and Martin, 2016). She expanded this paper to a mind-changing talk and argued that too much improvement work is undertaken in isolation at a local level, failing to pool resources and develop collective solutions. We need to have collaboration on system level, or we are just introducing new hazards in the process.

At the end of the first day, conference attendees were asked to write a job list for professionals from other disciplines, so that, for example, researchers can understand what patients need, and policy makers can understand what clinicians want. This is such a creative way to encourage listening and learning from each other. The era of collaboration has come! I am so inspired to go back to my job to implement what I have learned.