2015 Session Abstracts – Tuesday April 14th

Session E1 EBM into Practice
Tuesday April 14^th 11:00
Chair: David Nunan

Support for line managers to lead implementation of evidence-based practice – the importance of context 
Rebecca Mosson¹, Ulrica von Thiele Schwarz¹, Anne Richter², Henna Hasson^1
1Medical Management Centre, Karolinska Institutet, Stockholm, Sweden, ²Centre for Epidemiology and Community Medicine, Stockholm County Council, Sweden
Introduction: Leadership is essential when implementing evidence-based practice (EBP). Line managers have been identified as key actors for its successful implementation. However, managers do not lead and leadership does not take place in a vacuum. Thus, the context – conditions and factors that surround and affect an implementation effort – needs to be considered when investigating what support managers require to lead the implementation process.
Aim: The aim of this study was to investigate what contextual factors support line managers to lead implementation of EBP, and how these factors interrelate.
Methods:  Interviews were performed with 28 line managers within social care services in seven Swedish municipalities (the local authorities responsible for provision of these services). A purposeful sampling was performed to ensure diversity among the municipalities in terms of size, geographical location and previous experience with working according to EBP. Content analysis was performed. The Consolidated Framework for Implementation Research (CFIR) was used to conceptualize contextual factors that support managers to successfully implement EBP.
Results:  Preliminary results indicate that the managers mostly stressed factors related to the inner context as important, such as the organizational culture and climate, employees’ readiness for change, higher management, size and location of the organization, resources and access to new knowledge. In the outer setting, contextual factors such as clients’ needs and policy regulations were reported as greatly influential. Findings also showed that the interrelation between several contextual factors provided managers with optimal support.
Conclusion:  Although line managers’ have a central role in implementing EBP, they are greatly dependent on their context to succeed. The function of a combination of contextual factors may offer the most optimal support. By providing managers with support throughout the implementation process results in professionals working in the most optimal way. Consequently, clients’ will receive the best possible care.

Contextually relevant evidence to inform practice: A study of knowledge sharing at the inpatient-community care transition points in mental health.
Nicola Wright¹, Emma Rowley¹, Justin Waring¹, Arun Chopra², Kyriakos Gregoriou^2
1University of Nottingham, Nottingham, UK, ²Nottinghamshire Healthcare NHS Trust, Nottingham, UK
Introduction: Randomised Controlled Trials (RCT) provide high quality evidence of intervention effectiveness. However, this can come at a cost of understanding the context in which care is provided. For complex issues such as being admitted or discharged from inpatient mental health units, the relative simplicity of an RCT cannot capture the “messy”, disjointed and inefficient process which relies on cross-agency and organisational working.
Aims: The aim of the project was to explore the processes associated with admission and discharge from a mental health inpatient ward to the community. Of particular interest were the challenges experienced with sharing knowledge at the transition point between inpatient and community care.
Methods: One inpatient ward was the focus for the study. Qualitative focus group interviews were conducted and a total 52 people participated. This included service users, inpatient nurses, health care assistants, consultant psychiatrists, community mental health practitioners and junior doctors. The data was analysed using conventional thematic techniques.
Results: A contradiction was found between what was deemed to be best practice and the reality of working at care transition points. Words such as chaotic, stressful and emotionally charged were used to describe this process. Two forms of knowledge sharing was evident – what was written down and also the verbal handover. Both processes were passive on the part of the recipient. This meant that there were often gaps in information provision or conversely duplication as it was not in the required format for the receiver.
Conclusions: Knowledge sharing between individuals and organisations is vital for the provision of high quality care. By conducting contextually relevant research it is possible to highlight the areas where intervention is needed and to co-produce what this should be with local stakeholders. Therefore, potentially improving the care experience for service users through the quicker implementation of evidence based practice.

Clinical practice guidelines and decision makers in Catalan health system: what can help in their implementation?
Carme Carrion¹, Marta Aymerich², Liliana Arroyo-Moliner⁴, Emília Sánchez³, Pedro Gallo⁴, Jordi Caïs⁴, Joan Gené^5
1TransLab Research group, Universitat de Girona, Catalonia, Spain, ²Health Scieces Department, Universitat Oberta de Catalunya (UOC), Catalonia, Spain, ³Blanquerna School of Health Science, Universitat Ramon Llull, Barcelona, Catalonia, Spain, ⁴Department of Sociology and Organisational Analysis, University of Barcelona, Barcelona, Catalonia, Spain, ⁵CS-CAPSE, University of Barcelona, Barcelona, Catalonia, Spain
Introduction: Clinical practice guidelines (CPGs) are defined as a set of recommendations based on scientific evidence and designed to assist both healthcare professionals and users to select the most suitable diagnostic and/or therapeutic options for addressing a specific clinical condition. The main obstacle CPGs need to overcome is how to ensure their recommendations are best applied in routine practice. It is therefore important to determine what hinders and facilitates their implementation, so as to subsequently instigate relevant organizational and training measures.
Setting: Decision makers in the Catalan public system are frequently responsible for CPGs implementation at macro and meso level. Getting to know better their perceptions and attitudes towards CPGs and its relevance in the frame of the health system will allow for improved development and mainly implementation strategies.
Objective: To identify barriers and facilitators in the implementation of CPGs as perceived by decision makers in public healthy system in Catalonia.
Design: Focus groups techniques were used in which participants were asked to discuss their thoughts on a number of key dimensions, including accessibility and knowledge of CPGs, content and format, dissemination strategy, importance of training, CPGs in the context of the organization and social context.
Main outcomes and results: Participants believe that main barriers are lack of leadership and management support, not having enough trained professionals and teaching methods, and clinical inertia. About enablers, they think that pressure from specific interest groups, mainly pharma industry, knowing the contra factual and using them as key management tools may help in their implementation.
Conclusions: To be of real use, professionals should be better trained about CPGs content and use, in order to take them into account as management tools.

Clinical Performance Indicators evidencing good clinical practice
Joanna Shaw, Rachael Fothergill, Hannah Salvidge
London Ambulance Service NHS Trust, London, UK
Introduction: The London Ambulance Service NHS Trust (LAS) is the UK’s busiest emergency ambulance service with over 3,500 frontline staff attending to over one million patients every year.
Aims: Ensuring good clinical practice and implementing change can be challenging; therefore this quality improvement project aimed to evidence best practice and areas for improvement, and feedback directly to staff.
Methods: In 2001, Clinical Performance Indicators (CPIs) were developed for Paramedic Team Leaders to provide direct clinical supervision to all frontline staff.  The CPI process allows Team Leaders to review patient clinical records and assess their compliance to evidence-based aspects of care.  Team Leaders use a novel online database that collates information and uses it to produce individualised feedback forms.  They then meet with staff to provide recognition for good clinical practice and discuss areas for improvement.
Results: The ability to share this evidence directly with staff has led to improvements in clinical practice in each of the eight areas studied since current records began in April 2006: General documentation (89%-97%); Cardiac arrest (89%-98%); Acute coronary syndrome (90%-97%); Difficulty in breathing (87%-97%); Glycaemic emergencies (94%-97%); Patients left at home (83%-97%); Stroke (94%-97%; introduced April 2010), and Mental health (85%-90%; introduced April 2012).
Conclusions: Ongoing monitoring and engagement of staff has led to improvement.  This is particularly effective due to the robust feedback mechanism.  In 2008 the CPI process was adopted for all ambulance services in England and was later used as the basis for the Department of Health Ambulance Clinical Quality Indicators, for national comparative purposes.

Occurrence and determinants of selective reporting of clinical drug trials: design and preliminary results of an inception cohort study
Cornelis A. (Sander) van den Bogert¹, Patrick C. Souverein¹, Cecile T.M. Brekelmans², Susan W.J. Janssen³, Manon van Hunnik³, Gerard H. Koëter², Hubertus G.M. Leufkens¹, Lex M. Bouter^4
1Utrecht University, Utrecht, Utrecht, The Netherlands, ²Central Committee on Research involving Human Subjects, The Hague, Zuid-Holland, The Netherlands, ³National Institute for Public Health and the Environment, Bilthoven, Utrecht, The Netherlands, ⁴VU University Medical Center, Amsterdam, Noord-Holland, The Netherlands
Introduction (Background): Responsible conduct of research implies that study results should be completely and adequately reported. We will present the design of a study that aims to investigate the occurrence and the determinants of selective reporting in an inception cohort of all clinical drug trials that were reviewed by a Dutch Institutional Review Board in 2007. Preliminary results will be available at the time of the conference.
Methods: In 2007, Dutch IRBs reviewed 622 clinical drug trials. For each trial, we assessed its stage of progress. We discriminated five intermediate stages and five definite stages. The intermediate stages of progress are: approved by an IRB; started inclusion; completed as planned; prematurely terminated; published as article. The definite stages of progress are: rejected by an IRB; never started inclusion; not published as article; completely reported; selectively reported.
We will investigate whether trial characteristics are associated with non-publication using bivariate and multivariable models. Furthermore, we use Cox regression models to identify trial characteristics associated with the time to publication.
We will score the percentage of trials with one or more of 10 predefined discrepancies between protocol and publications. We also investigate the association between trial characteristics and discrepancies for each individual discrepancy and for the total discrepancy score. We also investigate the association between discrepancies and the direction of trial conclusions.
Preliminary results: Among the 622 trials included, 272 (44%) reported results in one or more full articles.
Conclusions: By measuring both non-publication and selective publication, we will be able to evaluate simultaneously determinants of both these questionable research practices. Our study will contribute to evidence-based medicine by identifying ways to improve the reporting practices of clinical drug trials.

Issues in scaling up the implementation of an effective intervention (BeST) after a definitive randomised controlled trial.
Helen Richmond¹, Zara Hansen¹, Esther Williamson¹, David Davies², Sallie Lamb^1
1University of Oxford, Oxford, UK, ²University of Warwick, Coventry, UK
Introduction:  The notion that psychosocial approaches are needed for the management of persistent LBP is widely acknowledged. The Back Skills Training Trial (BeST; n=701) showed the clinical and cost-effectiveness of a cognitive behavioural approach (CBA) for persistent LBP (Lamb et al, 2010). There is sparse research concerning the implementation of a CBA into routine physiotherapy practice. Understanding physiotherapists’ experiences of implementing BeST are imperative to build successful implementation efforts and align care with the best available evidence in this debilitating and costly field of healthcare.
Aim:  To explore the experiences of physiotherapists implementing BeST.
Methods:  Semi-structured interviews were conducted with 11 NHS physiotherapists working in secondary care across a range of geographical locations. Participants had received training in the BeST intervention through an online training programme (i-BeST) as part of a larger RCT. Interview transcripts were analysed using a thematic analysis that drew on grounded theory.
Results:  A key theme centred on participants anxieties in implementing BeST. Participants found that the BeST patient assessment and group sessions contrasted with their usual practice. They were anxious that the intervention would not be successful due to its unfamiliarity and struggled to identify and select suitable patients. Moving from a didactic to a discursive style of treatment delivery was particularly difficult. Participants lacked confidence in the use of new cognitive skills and in verbalising the more cognitive content. They were also concerned about their scope of practice, particularly when referring to the use of exploratory questioning.
Conclusions:  These anxieties illustrate the multifactorial nature of implementing a CBA into physiotherapists’ routine clinical practice. Scalable strategies to provide physiotherapists with more comprehensive clinical support are needed to facilitate implementation. Researchers should consider whether the design of trial processes, such as patient selection, reflects routine clinical practice to facilitate successful implementation.

Session E2 Too much medicine/Overdiagnosis & Global Health & EBM
11:00 Tuesday  April 14th
Chair: Peter Gill

EVIDENCE NOT IN EVIDENCE FOR COMMERCIALLY DRIVEN NUTRITION INTERVENTIONS IN LOW AND MIDDLE INCOME COUNTRIES
Khurshid Talukder
Centre for woman and Child Health (CWCH), Dhaka, Bangladesh
Introduction (Background): Current global recommendations to reduce under five stunting and wasting include the promotion of food fortification, commercially produced micronutrients and ready to use foods (RUFs). UN and other international agency attempts at scaling up these product-based, industry driven interventions are serious threats to programmes for the promotion, protection and support of breastfeeding and home cooked complementary foods. There is an urgent need to scrutinise the evidence claims of these products so low and middle-income countries (LMICs) can implement nutrition programmes suited to their needs.
Aims: To review the literature avilable on product based solutions to global malnutrition
Methods: Medline and Google searches were carried out with the following keywords: malnutrition, nutrition, fortification, micronutrients, micronutrient powders, ready to use foods, ready to use therapeutic foods, RUTF, breastfeeding, complementary feeding. Journal articles, organisation reports and homepages were scrutinised to look for evidence for commercial products proposed for nutrition interventions in LMICs.
Results: There were very few meta-analyses to address the claims of international bodies that these products are effective and culturally appropriate. What few there were, were inconclusive about their effectiveness. On the other hand, the evidence for the implementation and further strengthening of infant and young child feeding (IYCF) intervention is more compelling.
Conclusions: LMICS need to invest heavily in IYCF interventions not only to decrease under five malnutrition and death but also to start reducing the burden of overweight/obesity and non communicable disease (NCDs) that are now the biggest causes of mortality.

Diagnosing serious infections in acutely ill children in ambulatory care: diagnostic accuracy of a Clinical Decision Tree and added value of a Point-of-Care C-reactive protein Test: Study Protocol and Preliminary findings.
Jan Verbakel¹, Marieke Lemiengre², Tine De Burghgraeve¹, An De Sutter², Dominique Bullens³, Bert Aertgeerts¹, Frank Buntinx^1
1Department of General Practice, KU Leuven, Leuven, Belgium, ²Department of Family Practice and Primary Health Care, Ghent University, Ghent, Belgium, ³Clinical Department of Paediatrics, University Hospitals Leuven, Leuven, Belgium, ⁴Paediatric Immunology, Department of Microbiology and Immunology, KU Leuven,, Leuven, Belgium, ⁵Research Institute Caphri, Maastricht University,, Maastricht, The Netherlands
Introduction: Acute illness is the most common presentation of children to ambulatory care. In contrast, serious infections are rare and often present at an early stage. To avoid complications or death, early recognition and adequate referral are essential. In a recent large study children were included prospectively to construct a symptom-based decision tree with a sensitivity and negative predictive value of nearly 100%.
Aim: Aiming to improve detection of serious infections in ambulatory care, and reduce the number of false positives, point-of-care tests might be useful, providing an immediate result at bedside. The most probable candidate is C-reactive protein. We aimed to include over 6,500 children.
Methods: This is a diagnostic accuracy study of signs, symptoms and point-of-care tests for serious infections in ambulatory care. Acutely ill children presenting to a general practitioner or paediatrician were included consecutively in Flanders, Belgium. Children testing positive on the decision tree got a point-of-care C-reactive protein test. Children testing negative, randomly received either a point-of-care C-reactive protein test or usual care. The outcome of interest was hospital admission more than 24 hours with a serious infection within 10 days after first contact. We report the diagnostic accuracy of the decision tree (+/- the point-of-care C-reactive protein test) in sensitivity, specificity, and positive and negative predictive values. New diagnostic algorithms were constructed through classification and regression tree and multiple logistic regression analysis, if applicable.
Results: From 15 February 2013 to 28 February 2014, 8958 children were consecutively included by 192 general practitioners and 75 paediatricians throughout Flanders in 92 GP surgeries and 12 hospital departments. We identified 469 serious infections.
Conclusions: We will present a practical tool for diagnostic triage of acutely ill children in ambulatory care. We also aim to reduce the number of investigations and admissions in children with non-serious infections.

Objectification of diagnosis in cases with disseminated pulmonary lesions
Elena Filatova¹, Sofia Markina³, Igor Chernyaev², Sergey Skorniakov¹, Igor Medvinsky¹, Alexander Savelev¹, Igor Ganiyarov³, Nikita Zaitsev³, Olga Fedorova¹, Dmitry Repin^1
1Ural Research Institute for Phthiziopulmonology, Yekaterinburg, Russia, ²Sverdlovsk Regional TB Dispensary, Yekaterinburg, Russia, ³Ural Federal University, Yekaterinburg, Russia
Introduction: Disseminated pulmonary lesions (DPL) remain a clinical problem. The complexity is presented as the definition of the etiology of the disease, as well as an objective assessment of lesion volume to describe the dynamics of the pathological process under the influence of treatment and without it. The fact is that when there are multiple elements in the lungs visually difficult to count the number of elements and evaluate radiological improvement, the pictures look almost identical, even for experienced professionals.Aim To obtain an objective method for the quantitative calculation of the volume of lung tissue damage in cases of DPL.
Methods: Through the clinic UNIIF over the past 5 years have passed 489 patients with disseminated pulmonary lesions (TB, sarcoidosis and etc.), which were recorded in the database. Of these, we selected 30 of the most revealing cases with significantly confirmed diagnosis, each of which has been fully examined by clinical, laboratory and CT methods. We used special developed patented application that works with DICOM package, obtained during the CT scan, that allowed us to build an individual 3-D model of lungs with pathological elements for each patient, and then calculate the total volume of the lesions in dynamics: before, after or without treatment. Then DICOM packages of each case were given for evaluation to radiology expert. Reducing the total volume of pathological elements considered positive dynamics, increase – negative, a constant volume – as the lack of dynamics.
Results: In 80,3% of cases the assessment made using the application and meaning of the expert matched. Evaluation of radiographic also correlated with clinical data.
Conclusions:  Thus we get the new method for objective evaluation of the volume of lung tissue damage in cases of DPL at the stage of diagnosis and in dynamics during treatment with or without it.

Explaining the likelihood ratio; building up from basics or dissecting from the top; what works better??
veerle langenhorst, jolita bekhof
isala hospital, zwolle, The Netherlands
Introduction: explaining the likelihood ratio is a huge challenge; on average doctors are not much aware of judging the strength of a diagnostic test to in- or exclude a certain diagnosis. They tend to overvalue the strength of tests and they are frequently unaware of the pre test probability of disease. The likelihood ratio is a means to translate the pre test probability of disease into the post test probability of disease. More and more research is done to define likelihood ratio’s of certain tests. It’s of great importance to increase doctors’s awareness of the use of likelihood ratio’s in terms of quality of care and cost effectiveness.
Aims: we investigated 2 different teaching methods to explain likelihood ratio’s with the aim to measure the increase in knowledge before and after the teaching session
Methods: 2 well experienced EBM teachers performed 2 different teaching strategies; one (group 1) was starting from basics explaining sensitivity, specificity, positive and negative predictive values and from there working towards the likelihood ratio and nomogram. The other method (group 2) started with the likelihood ratio and nomogram and worked ‘backwards’ dissecting it gradually from the top ending with the sensitivity, specificity, positive and negative predictive values. We asked all participants to rate their baseline knowledge about the likelihood ratio as basic, moderate or good. We asked them before and after the teaching session to plot on a 10 cm visual analogue scale how familiar they felt with the likelihood ratio; 0 meant not familiar at all, never heard of. 10 meant very familiar, able to teach somebody else.
Results: we included 60 medical doctors. …% was registrar pediatrics, …% was consultant, …% was rest group. baseline knowledge judged by themselves was ….% basic, …% moderate, …% good. the rest of results will follow
POSTER – Describing knowledge encounters in healthcare: a systematic mixed studies review and development of  a classification
POSTER – Investigating mood instability using natural language processing (NLP) – a novel, automated method to extract data from electronic health records

Views and experiences of rural doctors on evidence based medicine: the FrEEDoM qualitative study
Ranita Hisham¹, Su May Liew¹, Chirk Jenn Ng¹, Paul Glasziou², Kamaliah Mohd Nor³, Iskandar Firzada Osman⁴, Gah Juan Ho¹, Nurazira Hamzah^1
1University of Malaya, Kuala Lumpur, Malaysia, ²Bond University, Queensland, Australia, ³Ministry of Health, Putrajaya, Malaysia, ⁴Jaya Gading Health Clinic, Pahang, Malaysia
Background: Evidence-based medicine, the integration of individual clinical expertise, best external evidence and patient values, was introduced more than a decade ago. Yet, primary care physicians still experience problems with using evidence in clinical practice.
Aim: To explore the views and experiences of primary care physicians about evidence-based medicine in their daily clinical practice at rural primary care settings.
Methods: A qualitative study design was used. Participants were recruited by purposeful sampling. Interviews were conducted in June 2013 in two rural health clinics, Karak and Maran Health Clinics, Pahang, Malaysia. Four focus group discussions with 15 medical officers and 3 individual in-depth interviews with family medicine specialists were carried out. All interviews were conducted using a topic guide and were audio-recorded, transcribed verbatim, checked, and analysed using a thematic approach.
Results: Key themes identified were 1) doctors viewed evidence-based medicine as being mainly statistics, research and guidelines (2) reactions to evidence-based medicine were largely negative (3) doctors relied on specialists, peers, guidelines and non-evidence based internet sources for information (4) information sources were accessed using novel methods such as mobile applications (5) there were several barriers to evidence-based practice, including doctor-, evidence-based medicine-, patient- and system-related factors. These included inadequacies in knowledge, attitude, management support, time and access to evidence-based information sources. Participants recommended the use of online services to support evidence-based practice in the rural setting.
Conclusion: The level of evidence-based practice is low in the rural setting due to poor awareness, knowledge, attitude and resources. Doctors are using non-evidence based sources accessed through new methods such as messaging applications. Further research is recommended to develop and evaluate interventions to overcome these identified barriers.

Implementing Evidence Based concepts in developing countries: Challenges and achievements at Cairo University.
Ahmed Elkhadem
Faculty of Oral & Dental Medicine, Cairo University, Cairo, Egypt
Introduction: Education at Faculty of Oral and Dental Medicine, Cairo University is divided into under-graduate and post-graduate. Postgraduate education involves Master and PhD degrees for three academic departments and nine clinical ones with total new enrollments reaching up to 400 post-graduate students per year. This huge number of candidates was supposed to produce an equivalent number of international publications, but that was not the case. Two of the reasons behind the decreased number of international publications was the inadequate trial performance and poor reporting. The concept of “randomized controlled trial” was not familiar neither to post-graduate students nor their supervising staff members. Hence, no single RCT was performed at the faculty level before the year 2010.
Aim: To enhance trial performance and reporting by following reporting guidelines published on Equator network.
Materials and methods: A Center for Evidence Based Dentistry (EBD) was established in June 2010 with the aim of increasing awareness between faculty members and post-graduate students to different types of study designs and trial reporting guidelines. An EBD taskforce was formed from seven faculty members. The steps of taskforce recruitment and training are discussed. The challenges facing this taskforce are highlighted.
Results: Tables and graphs will be presented to show the impact of implementing evidence based knowledge. A paradigm shift from invitro to randomized controlled trials occurred. Systematic reviews protocols using PROSPERO guidelines have been accepted for the first time in 2013. The process of integrating evidence based science into post-graduate curricula is shown.
Conclusion: Resistance to change is the biggest challenge facing implementation of evidence based reporting guidelines at Faculty of Oral & Dental Medicine – Cairo University. Continuous learning, perseverance and team work are the pivots for success.

Undergraduate medical students’ perceptions, attitudes, and competencies in evidence-based medicine (EBM), and their understanding of EBM reality in Syria
Ammar Sabouni¹, Fares Alahdab¹, Belal Firwana², Rim Hasan², Mohamad Bassam Sonbol¹, Munes Fares¹, Iyad Alnahhas¹, Mazen Ferwana^3
1Faculty of Medicine, Damascus University, Damascus, Syria, ²Department of Internal Medicine, University of Missouri, Columbia, MO, USA, ³National and Gulf Center for Evidence Based Health Practice, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
Background: Teaching evidence-based medicine (EBM) should be evaluated and guided by evidence of its own effectiveness. However, no data are available on adoption of EBM by Syrian undergraduate, postgraduate, or practicing physicians. In fact, the teaching of EBM in Syria is not yet a part of undergraduate medical curricula.
Aims: The authors evaluated education of evidence-based medicine through a two-day intensive training course.
Methods: The authors evaluated education of evidence-based medicine through a two-day intensive training course that took place in 2011. The course included didactic lectures as well as interactive hands-on workshops on all topics of EBM. A comprehensive questionnaire, that included the Berlin questionnaire, was used to inspect medical students’ awareness of, attitudes toward, and competencies’ in EBM.
Results: According to students, problems facing proper EBM practice in Syria were the absence of the following: an EBM teaching module in medical school curriculum (94%), role models among professors and instructors (92%), a librarian (70%), institutional subscription to medical journals (94%), and sufficient IT hardware (58%). After the course, there was a statistically significant increase in medical students’ perceived ability to go through steps of EBM, namely: formulating PICO questions (56.9%), searching for evidence (39.8%), appraising the evidence (27.3%), understanding statistics (48%), and applying evidence at point of care (34.1%). However, mean increase in Berlin scores after the course was 2.68, a non-statistically significant increase of 17.86%.
Conclusion: The road to a better EBM reality in Syria starts with teaching EBM in medical school and developing the proper environment to facilitate transforming current medical education and practice to an evidence-based standard in Syria.

Session E3 public and patient involvement
11:00Tuesday April 14th
Chair: Paul wicks

Written information for patients on the use of antibiotics in acute upper respiratory infections in primary care
Jack O’Sullivan, Robert Harvey, Paul Glasziou, Amanda McCullough
Bond University, Gold Coast, QLD, Australia
Introduction: Inappropriate antibiotic use drives antibiotic resistance. Antibiotics have little or no benefit for upper respiratory tract infections (URTIs) yet they continue to be prescribed. We sought to systematically review the effect of written patient information about antibiotics for URTIs on subsequent antibiotic use for URTIs.
Methods:  We sought randomised controlled trials of written patient information (both paper and electronic) about antibiotics for URTIs given at the time of consultation in participants of any age defined as having an URTI. We searched CENTRAL (comprised of 6 databases) from inception to September 2014.  Two review authors independently screened returned articles, extracted data (number and type of participants, description of the intervention and antibiotic use subsequent to the index consultation) and assessed risk of bias using the Cochrane Risk of Bias tool.  Any disagreements were resolved by a third author.
Results:  Of 315 studies identified, five studies were included (15,390 participants, adults with URTI [n= 14669], parents of children with URTI [n= 751]). Data extraction and synthesis is ongoing. The interventions included leaflet (n=2), posters (n=1), booklets (n=1) and brochures (n=2) (*Note: Some studies used a combination of interventions). Two studies measured our primary outcome of antibiotic use following the intervention. Risk reductions varied but were in favour of the intervention group: leaflet (absolute risk reduction [ARR]=15%, relative risk reduction [RRR]=24%) and booklet (ARR=21%, RRR=48%). Both of these studies were assessed as being at low risk of bias. The risk of bias of the remaining three studies was low (n=2) and unclear (n=1).
Conclusion:  Preliminary evidence suggests that written information could be an effective tool to reduce antibiotic use for URTIs. Meta-analysis of these studies is needed to assess whether this effect is sustained when these studies are synthesised.

 A Just In Time Personalised Evidence Service
Muir Gray
NHS England, London, UK
Background: NHS Choices gets 40,000,000 visits a month but needs activating
Aim: I would like to discuss the proposed development of an Evidence and Knowledge Service for citizens and patients.  The proposal has four stages but we would concentrate principally on stages 3 and 4 on the assumption that Stages 1 and 2 will be covered by other conference speakers.
Stage 1. Production and Organisation of Best Current Knowledge. At present this is collected in NHS Choices, steps have been taken to produce a stronger quality assurance process based on the Information Standard.
Stage 2. The presentation of data using evidence based methods to minimise bias, for example those developed by Gerd Gigerenzer and Woloshin and Schwartz.
Stage 3. At present the NHS England sends out 1.5 billion letters, 1 billion lab reports and a billion prescriptions every year and Evidence could be embedded in all of these communications
As part of the policy called Personalising and Health and Care 2020 these will all be turned into digital communications bearing in mind the fact that some groups of the population are not able to use this medium.  The aim is to send evidence just in time namely when a person
– is considering a test
– receives a test result
– is considering an intervention of some sort
–  is waiting for an intervention such as colonoscopy or an operation
–  has had a hospital admission or intervention
–  when a person is living with a chronic condition
Stage 4.  Implications of these changes for the consultation and the medical profession
Methods: We would like to consult before the conference as well as speaking at it
Conclusion: The conclusion will shape the development of this ‘Just in Time’ Evidence Service

Clinical Trials, Risk and Rescue: Are Information Pathways Sufficient?
Amy Price¹, Su May Liew², Amanda Burls³, Jazmin Price⁴, Taylor Lopreto⁵, Jo Kirkpatrick^6
1University of Oxford & ThinkWell, Oxford, UK, ²University of Malaya, Kuala Lumpur, Malaysia, ³City University London & ThinkWell, London, UK, ⁴Empower2Go & ThinkWell, Delray, Florida, USA, ⁵Empower2Go & ThinkWell, Delray, Florida, USA, ⁶Empower2Go & ThinkWell, Delray, Florida, USA
Introduction: Clinical trials are advertised as safe pathways to innovation in healthcare. Marketers cite advances in technology and plain language as markers of successful participant experience in clinical trials even as recruitment, compliance and completion rates are dropping. We ask participants what they perceive to be strengths, weaknesses, threats and opportunities in clinical trials.
Aims: To explore practice breakdowns in clinical trials pathways and suggest repairs as proposed by the public through interviews and social media channels.
Methods:  Researchers and volunteers independently coded publicly available interviews, social media conversations and Twitter hash tags streams about clinical trials into predetermined themes of risk and rescue around clinical trials. Analysis of the data was mapped through focus groups and agreement reached through online review.
Results:  The study found that information transfer regarding risk and rescue was inadequate with significant gaps in participant protections. Participants recommend consistent information reporting within a trial, extensive information access restructuring, better risk communication and pre-determined compensation guidelines for participants injured as a result of research interventions such as the posting of a refundable bond by research sponsors.
Conclusions:  Public conversations concerning clinical trials move beyond requests for plain language and revolve around practical solutions. Some participants share vision to be proactive was nurtured though the All Trials campaign where they learned that all trials must be registered and reported to build a strong foundation for future research and current medical practice. The public calls for high methodological standards, safety, understanding of risk and fiscal responsibility in exchange for their full participation in clinical trials.

Miracle Cures and Health Scares: taking the people to the evidence behind the headlines
Andy Gibson, Kate Boddy, Kath Maguire, Jo Welsman
University of Exeter Medical School, Devon, UK
Introduction: ‘Mobile phones cause brain cancer’; the media is constantly full of claims about the potential health benefits or risks associated with particular treatments, foods or substances. How do we enable people to make sense of the evidence behind the headlines?
Aims: We discuss the development of a workshop which makes use of media stories to introduce lay audiences to the basic ideas behind evidence-based medicine (EBM).
Methods: Working with the Peninsula Public Involvement Group, the patient and public involvement team at the South West Peninsula Collaboration for Leadership in Applied Health Research and Care have designed ‘Health Scares and Miracle Cures’ workshops. These workshops use newspaper stories about health to explore some of the core ideas of EBM. Sessions include ‘Why do we need research?’ and ‘Searching for evidence’. The workshop can be run in a 1.5 hour or 3 hour format. The resources required are minimal and can be delivered in a variety of settings, for example village halls.
Results: Between 2011 and 2014 the team delivered 16 of these workshops to 215 people across a wide age range (16 – 80+ years), social backgrounds and organisations such as Local Involvement Network (LINk) members, Housing Association residents, ex-offenders and retired people. Notably the workshop has been delivered 7 times to a total of 99 ‘A’ level students (aged 16-18 years) and have been positively evaluated by participants. For some organisations these workshops have helped them introduce members to concepts such as evidence based policy. On other occasions they have opened the gate way for individuals to become more involved in health research or helped them develop skills in critical thinking.
Conclusions: We have demonstrated the feasibility of presenting basic scientific concepts to a wide variety of lay audiences in different settings using a flexible and accessible format.

Patient and public involvement in research question generation and prioritisation: A view from the NIHR CLAHRC for the South West (PenCLAHRC)
Jo Welsman¹, Andy Gibson¹, Kath Maguire¹, Heather Boult², Nigel Reed², Darren Moore^3
1University of Exeter, Exeter, UK, ²Peninsula Public Involvement Group, Exeter, UK, ³Evidence Synthesis Team, NIHR CLAHRC for the South West (PenCLAHRC), Exeter, UK
Background:  Traditionally, academics have driven the research agenda within health services research, but our organisation is working collaboratively with the public to ensure that “the simple but important questions get answered” (PPI participant).  This approach aims to reduce wastage in research and to deliver results which more readily translate into treatments acceptable to patients.
Aims:  To discuss how the PenCLAHRC Public Involvement team involve the public in research question generation and prioritisation and highlight the impact this involvement has on the questions prioritised for development into funded research.
Methods:  In September 2014 we ran 9 workshops throughout the Peninsula with participants recruited from public involvement networks and the Peninsula Public Involvement Group (PenPIG). Participants generated broad research themes and PICO (population, intervention, comparator, outcome) was used to structure these into research questions subsequently submitted via an online submission tool. Prioritisation involves three voting rounds in which PenPIG are an equal stakeholder alongside academic and NHS partners. Their involvement in this process, as individuals via email, as a group, and as representatives at the final stakeholder meeting will be discussed.
Results:  60 workshop attendees generated 106 research themes. Mental health was a priority (21% of themes), followed by diabetes (8%) and long term conditions in youth (7%). Overall, 72 questions were submitted with 25 (35%) from public contributors, 36 (50%) from Clinicians and 11 (15%) from Academics. Of the 9 questions voted through to the final stakeholder prioritisation meeting, 4 came from public contributors, and 3 had mental health themes.
Conclusions:  The proportion of questions and the research priority identified in initial workshops by public participants were retained throughout the PenCLAHRC prioritisation process. This highlights the significance and relevance of public and patient generated questions to the research agenda of a major organisation delivering health services research.

Session E4 Improving conduct of research
11:00 Tuesday April 14th
Chair: Jeremy Howick

Rapid Review Programs to Support Health Care and Policy Decision Making: A Descriptive Analysis of Processes and Methods
Julie Polisena¹, Chantelle Garritty², Chris Kamel¹, Adrienne Stevens², Ahmed Abou-setta³, Valerie King^4
1Canadian Agency for Drugs and Technologies in Health, Ottawa, Canada, ²Ottawa Hospital Research Instititute, Ottawa, Canada, ³University of Manitoba, Winnipeg, Canada, ⁴Oregon Health Sciences University, Portland, USA
Background: Health care decision makers often make decisions under limited timeframes that do not allow the completion of a full evidence reviews. Rapid reviews (RRs), utilizing streamlined systematic review methods, are increasingly being used for evidence synthesis to support such decisions.
Aims: Our primary objective was to describe the processes and methods used across international programs to produce RRs. We also sought to understand the underlying themes associated with these programs.
Methods:  We contacted representatives of international health care RR programs to solicit information about the methods and processes used to produce RRs. The responses were summarized narratively to explain the characteristics associated with each program’s processes and methods. The summaries were compared and contrasted to highlight potential themes and trends related to the developing science and practice of RRs.
Results: Twenty-nine RR programs were included in our sample with an extensive representation from academia, government, research institutions, and non-for-profit organizations.  Responses revealed that the main objectives for RRs were to inform decision making on funding health care technologies, services and policy, and program development. Central themes that influenced the methods used, and report type and dissemination were the needed report timeline, resources available, the complexity and sensitivity of the research topics, and proprietary nature of the product.
Conclusions: Our study revealed that there is no standard approach to conduct RRs. Differences in processes and methods across programs may be the result of the novelty of RR methods versus other types of evidence syntheses, customization of RRs for various decision makers, and definition of ‘rapid’ by organizations. Future research should investigate the impact of RR methods and reporting to support informed health care decisions, the effects of potential biases that may be introduced with streamlined methods, and the effectiveness of RR reporting guidelines on transparency.

Applications and Appropriateness of Rapid Reviews
Valerie King¹, Julie Polisena², Chantelle Garritty^3
1Center for Evidence-based Policy, Oregon Health & Science University, Portland, Oregon, USA, ²Canadian Agency for Drugs and Technologies in Health (CADTH), Ottawa, Ontario, Canada, ³Ottawa Hospital Research Institute (OHRI), Ottawa, Ontario, Canada
Introduction: Both clinical and health policy decision-makers are often constrained by time and resources in developing guidelines, policies and programs. The time required for traditional systematic reviews (SR) is generally insufficient to inform rapidly evolving policy needs and some organizations lack the financial and staff resources even when time is not critical. Rapid reviews (RR) are an attempt to meet decision-maker needs within available resources. However, there is not a single standard for RRs and they may not be appropriate in all circumstances.
Aims: To describe the ways RRs are developed and used to inform clinical and health policy and their appropriateness for these purposes.
Methods: Critical case studies of RR programs across three North American research organizations.
Results: We profile three RR producers: two Canadian and one U.S.-based–the Ottawa Health Research Institute (OHRI), the Canadian Agency for Drugs and Technologies in Health (CADTH), and the Center for Evidence-based Policy (CEBP). These programs collectively produce over 500 RRs annually, using streamlined processes for synthesizing evidence. Products are generally developed using methods customized to meet client needs and completed within weeks to months. We present real world cases illustrating how decision makers have used RRs to support evidence-informed health decisions: OHRI’s on the use of personal protective equipment (PPE) for health care workers caring for patients with Ebola was used to inform the WHO PPE Guidelines; CADTH’s on pre-operative skin preparation informed clinical recommendations to standardize practice; and CEBP’s on newer drugs to treat chronic hepatitis C aided multiple states developing coverage criteria.
Conclusions: The RR approach to evidence synthesis is generally well-accepted by public decision makers and useful to help inform policies. While RRs may be less appropriate when the question requires high levels of methodologic rigor, transparency can help to inform users and the public about potential limitations.

Prioritising research for Chronic Kidney Disease in cats: adapting the James Lind Alliance method to a new clinical setting
Rachel Dean¹, Katherine Cowan², Marnie Brennan¹, Sharon Mickan^3
1Centre for Evidence-based Veterianry Medicine, University of Nottingham, Loughborough, UK, ²Katherine Cowan Consulting, St Leonards on Sea, UK, ³Centre for Evidence-based Medicine, University of Oxford, Oxford, UK
Introduction: Chronic Kidney Disease (CKD) is a significant cause of morbidity and mortality in older cats. Currently there is some evidence available about the efficacy of some of the multiple interventions available to manage this disease, but much uncertainty remains.
Aims: The aim of this study was to identify the research priorities of practising veterinary surgeons and cat owners about the efficacy of treatment options for CKD. The secondary aim was to see if the priority setting partnership (PSP) framework used by the James Lind Alliance could be adapted and applied to the veterinary setting.
Methods: An online survey was circulated through a number of routes to make contact with veterinary surgeons and cat owners with experience of CKD in cats.  Through this survey a long list of questions about treatments for CKD was collated. The evidence-base was searched to identify which questions were ‘treatment uncertainties’ and which were ‘known unknowns’. A short list of questions, true treatment uncertainties, was then prioritised in a workshop of veterinary surgeons and cat owners.
Results: Three hundred veterinary surgeons and cat owners responded to the initial survey and provided 649 questions; 386 were about treatment. Over 1/3 of the treatment questions were non-specific to a single therapy and there was much repetition. Twenty eight treatment uncertainties were identified and used in the PSP workshop. The 9 veterinary surgeon and 4 cat owners agreed on a shared list of prioritised questions regarding the treatment of CKD.
Conclusions: Prioritised treatment uncertainties were identified for CKD in cats. This study demonstrates that the JLA framework can be successfully adapted to a new healthcare setting, further integrating EBM methodology to new areas.

IDEAL: No innovation without evaluation – changing health care systems’ approach to surgical procedures and medical devices
Daphne Austin¹, Muir Gray¹, Allison Hirst², Peter McCulloch^2
1Rightcare, London, UK, ²Oxford University, Oxford, UK
It has been the cultural norm in Western health care systems to bring certain types of new treatments into use as ‘routine care’ without an adequate evidence base. Worse, once introduced into routine practice, the development of strong evidence through evaluation may be resisted. This is highly wasteful of health care resources and exposes patients to unacceptable risks.
The IDEAL (Idea, Development, Exploration, Assessment, Long-term follow-up) Framework (http://www.ideal-collaboration.net/) sets out a framework for generating evidence for surgical procedures during innovation in a rational and progressive way. The model offers healthcare systems a development path for surgical innovation that parallels that internationally adopted for medicines.  The framework can also be used for medical devices.
IDEAL has been available since 2009 (McCulloch et al) but as yet has barely registered with clinicians, those assessing evidence, policy makers and funders of health care.  Yet the implications of the framework are profound.
This paper presents the case that it is unethical for health care systems to innovate without evaluation; sets out the widespread benefits that could be delivered by embedding IDEAL at the heart of research, clinical and healthcare funding practice; and finally points to the cultural changes required to consistently deliver evidence based surgical procedures and medical devices.
McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, Nicholl J, for the Balliol Collaboration, No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009; Sep 26; 374 (9695): 1105-12.

Why evidence-based medicine needs ‘systemic independence’
Huw Llewelyn
Aberystwyth University, Aberystwyth, UK
Introduction: Sensitivity, specificity and likelihood ratios for use in Bayes theorem with statistical independence currently form the basis for ‘evidence-based’ diagnosis.
Aims: To show that unless the performances of tests are also assessed for use with reasoning by elimination and ‘systemic’ independence’ the usefulness of many tests will be overlooked.
Methods with worked example: If localised right lower quadrant (LRLQ) pain occurs in 50% of those with appendicitis and 50% of those without appendicitis, the likelihood ratio is 1 and it seems unhelpful.  Similarly, if guarding occurs in 50% of those with appendicitis and 50% of those without appendicitis, the likelihood ratio is 1 and guarding also seems to be unhelpful.  They will also seem unhelpful if used in combination as the combined likelihood ratio assuming statistical independence is 1 x 1 = 1.  However, assume that 50% of those without appendicitis had ‘non-specific abdominal pain’ (NSAP) and they all had LRLQ pain, the others without appendicitis or NSAP never having LRLQ pain. Also assume that of those without appendicitis who had NSAP, none had guarding.  This means that if a patient has LRLQ pain, he or she must have appendicitis or NSAP.  If the patient has guarding then as this never occurs in NSAP but often occurs in appendicitis, the diagnosis must be appendicitis.  So despite all the likelihood ratios being 1, the combination of LRLQ pain and guarding predict appendicitis with certainty.  This is how reasoning by elimination and ‘systemic independence’ between LRLQ pain and guarding works.
Results: The effect of assessing data giving various proportions of greater than 0% and less than 100% will be described using statistical and systemic independence.
Conclusion: Failure to take into account ‘systemic’ independence could result in many useful tests being overlooked, including when they are being assessed for stratified medicine.

EBRNetwork – A call to action for more (efficient) systematic reviews
Hans Lund¹, Matt Westmore², Klara Brunnhuber³, Maureen Dobbins⁴, Karen Robinson⁵, Marlies Leenars⁶, Hanna Nykvist⁷, Robin Christensen⁸, Malcolm MacLeod⁹, Bertil Dorch¹⁰, Mona Nasser^11
1Evidence-based Research Network, Odense, Denmark, ²Evidence-based Research Network, Southampton, UK, ³Evidence-based Research Network, London, UK, ⁴Evidence-based Research Network, Hamilton, Canada, ⁵Evidence-based Research Network, Baltimore, USA, ⁶Evidence-based Research Network, Nijmegen, The Netherlands, ⁷Evidence-based Research Network, Bergen, Norway, ⁸Evidence-based Research Network, Copenhagen, Denmark, ⁹Evidence-based Research Network, Edinburgh, UK, ¹⁰Evidence-based Research Network, Odense, Denmark, ¹¹Evidence-based Research Network, Plymouth, UK
Introduction: Scientists are expected to refer to earlier research results when they argue for the need for a new study or summarise its results. But research on research shows that the chosen references are often insufficient and biased towards the interest of the researchers; that researchers are not supported (through funding, time, training) in the production and updating of systematic reviews; and that there is a need for new ways of conducting systematic reviews that are more efficient yet rigorous.
Aims: To address these problems, a group of Norwegian and Danish researchers have initiated an international network, the ‘Evidence-Based Research Network’ (EBRNetwork – www.ebrnetwork.org). With initial partners from Australia, the UK, USA, Canada, Denmark, the Netherlands and Norway, the Network was established in Bergen, Norway in December 2014.
Methods: We are presenting the findings of several relevant studies, including the use of previous research by scientists; the problem of ongoing research after a benefit or harm of an intervention has been unequivocally established; and an international comparison of research funders, regulators and publishers regarding policies mandating systematic reviews prior to new research. We issue an invitation to join the EBRNetwork and will work with the members of the audience toward identifying and prioritising key initial workstreams.
Results: The alarming findings of the identified research have shown that too much health and medical research is unnecessary, unethical, unscientific, and wasteful.
Conclusions: The new EBRNetwork is an international collaboration that aims to ensure that no new studies are approved, funded or published without systematic review of existing evidence; and works towards more efficient production, updating and dissemination of systematic reviews. The Network issues a call to participate in developing a consensus statement to accomplish these aims.

Session E5 Evidence for diagnostics
11:00 Tuesday April 14th
Chair: Phillip Turner

Are systematic reviews used in the planning and design of NIHR HTA trials? A retrospective cohort
Sheetal Bhurke¹, Andrew Cook², Anna Tallant¹, Amanda Young¹, Elaine Williams¹, James Raftery^2
1National Institute for Health Research (NIHR), Evaluation, Trials and Studies Coordinating Centre (NETSCC), University of Southampton, Southampton, UK, ²Wessex Institute, University of Southampton, Southampton, UK, ³University of Southampton and University Hospital Southampton NHS Foundation Trusts, Southampton, UK
Background: There is limited evidence on how systematic reviews are used in the design of new trials. A recent study (Jones 2013) investigated this using a cohort of randomised trials funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme during the period 2006-2008. The study found that 11 (23%) of 48 applications made no reference to a systematic review. Of the 37 trials referencing a systematic review 20 (54%) reported their use in designing the trial.
Aims: To replicate and verify Jones’ study as well as explore the reasons for trials not using a systematic review. The study also included a recent cohort (funded in 2013) of NIHR HTA trials to identify if there were improvements over time.
Methods: Two cohorts of NIHR HTA randomised controlled trials were included. Cohort I included the same trials as Jones et al (with the exception of one trial which was discontinued). Cohort II included NIHR HTA trials funded in 2013 only. Data extraction was undertaken independently by two reviewers. Results were presented using only descriptive statistics.
Results: Systematic reviews cannot always be used to justify the need for new research. Our study shows nine (19%) and three (9%) trials from cohort I and II respectively did not reference  a systematic review, but each had a justifiable reason. Approximately 27% of all trials used a different primary outcome to the previous systematic review.
Conclusions: Systematic reviews were referenced and used to inform the trial design in nearly 85% of NIHR HTA trials. NIHR now requires that all proposals for potential funding for new primary research is justified by existing evidence and our study confirms that this is happening in practice. However, systematic review authors could maximize the impact of future trials by reporting clinically relevant primary outcomes.

Impact of NIHR HTA Programme funded research on NICE clinical guidance: a cohort study
Sheila Turner¹, Sheetal Bhurke¹, Andrew Cook^2
1NETSCC, University of Southampton, Southampton, UK, ²Wessex Institute, Southampton, UK
Introduction: It is vitally important that there is a connection between health research and clinical practice. Indications as to the impact of the research on evidence-based practice and policy can be obtained by tracking the use of outputs of health research, especially its use in clinical guidelines. This study considers the relationship, over a time period of more than 10 years, between a major UK research funder, the National Institute for Health Research Health Technology Assessment (NIHR HTA) Programme and an organisation with an international reputation responsible for issuing national guidelines, the National Institute for Health and Care Excellence (NICE).
Aims: The aims of the study were to assess the percentage of NICE clinical guidelines (CGs) informed by NIHR HTA studies, and the impact of evidence from those studies on the included NICE CGs.
Methods: Firstly we assessed the percentage of NICE clinical guidelines (CGs) issued between April 2001 and April 2012 which cited evidence from studies funded by the NIHR HTA Programme. To assess the impact of evidence from NIHR HTA studies on NICE CGs, we developed a classification system based on the evidence contributed by NIHR HTA studies to the CGs.
Results: From a total of 122 NICE CGs 76% had either been based upon or utilised evidence from NIHR HTA studies. The impact of NIHR HTA evidence on NICE CGs varied in the type and quantity of data used.
Conclusions: Findings suggest that NIHR HTA funded research impacts substantially on clinical guidance from NICE and hence is well connected to both clinical practice and policy.

A plea for presenting prediction intervals in meta-analysis
Joanna IntHout, Jelle Goeman
Radboud university medical center, Research Institute for Health Sciences, Nijmegen, The Netherlands
Introduction (background): In random-effects meta-analyses a confidence interval (CI) provides information on the effect size average of an intervention, but not on its variation. For many readers the clinical consequences of heterogeneity when expressed by I² or τ² are not immediately clear, especially if the meta-analysis is on odds ratios or risk ratios.
Aims: We aim to show the usefulness of prediction intervals for clinical decision making. A prediction interval represents the heterogeneity on the same scale as the outcome and is therefore well suited to evaluate the variability of the effect of an intervention.
Methods: Conclusions based on CIs may not hold in all settings. We evaluated the discrepancy in conclusions based on 95% CIs and 95% prediction intervals in statistically significant meta-analyses with heterogeneity (I²>0) of the Cochrane Database Issues 2009-2013.
Results: A statistically significant meta-analysis does not guarantee that the treatment will be effective in all settings: in 347 (72%) of the 479 statistically significant meta-analyses with I²>0, the prediction interval showed that the treatment could be ineffective. We show how to calculate the probabilities of these events.
Conclusions: The CI is inadequate for clinical decision making because it only summarizes the average treatment effect. The prediction interval is more informative as it shows the range of possible effects in relation to the no-effect and clinical benefit thresholds. A narrow prediction interval completely on the beneficial side of a clinically relevant threshold increases confidence in an intervention. A broad prediction interval may indicate the existence of settings where the treatment has a suboptimal and possibly even harmful effect. This situation calls for an investigation of the possible causes of the heterogeneity.
Not knowing the prediction interval results in a waste of useful information. The prediction interval should be reported and used in clinical decision making.

Reporting requirements for systematic reviews by medical specialty and publishing group
Haya Almalag¹, Arduino Mangoni², Mike Crilly^1
1Aberdeen University Medical School, Scotland, UK, UK, ²Flinders University, Adelaide., Australia
Background: The ‘Preferred Reporting Items for Systematic reviews and Meta-Analysis’ (PRISMA) and ‘Meta-analysis Of Observation Studies in Epidemiology’ (MOOSE) are the two major guidelines available to support the reporting of systematic reviews of clinical trials and observational studies in the medical literature.
Aim: To investigate the recommendation of PRISMA/MOOSE by medical journals indexed on the Thomson-Reuters ‘Journal Citation Report’ (JCR).
Methods: We selected the 20 highest ‘impact factor’ journals from the 34 medical specialties/catergories indexed on JCR and obtained their ‘instructions for authors’. A single investigator assessed whether PRISMA/ MOOSE were recommended by these journals, along with any mention of the EQUATOR network and Cochrane Collaboration. Impact factor was summarised as median (IQR) with analysis of the percentage recommending PRISMA/ MOOSE by medical specialty and publishing group.
Results: We reviewed the ‘instructions to authors’ for 603 journals, among which 49 (8%) contributed to more than one medical specialty. Overall 101 (17%) journals recommended the use of PRISMA/ MOOSE; 96 (16%) recommended PRISMA and 48 (8%) MOOSE. Guidance only rose to 108 (18%) with the inclusion of reference to EQUATOR (36 journals) and the Cochrane Collaboration (10 journals). MOOSE/ PRISMA recommendation was highest for obstetrics/gynaecology (55%), general/internal medicine (45%), orthopaedics (35%) and cardiology (30%). Recommendation in public health related journals was surprisingly low (10%) and was completely absent for pharmacology/pharmacy and ophthalmology. Among 16 major publishers (>= 5 journals) PRISMA/ MOOSE recommendation was highest for BMJ (42%), JAMA (33%), Stanford (31%) and Oxford (25%) University. Recommendation in Nature publications was low (3%) and completely absent from journals published by Annual-Reviews, Thieme, Mary-Ann-Liebert, Informa and Karger. We found no statistical association between the median impact factor and recommendation of PRISMA/MOOSE by medical speciality or publishing group.
Conclusion: Reporting guidelines for systematic reviews are currently endorsed by relatively few journals and their publishers.