In 1995 Sterling reported nothing had changed in the last 30 years when it came to publication bias: in his 1959 study of the four major psychology journals publications he reported that 97% of published studies were statistically significant.
A 2010 HTA systematic review reported that half of all trials had never published results, and positive trials were twice as likely to be published than negative trials.
In 1994 researchers commonly used the wrong techniques, or the right techniques wrongly, misinterpreted results, reported results selectively and often came to unjustified conclusions. All of this meant “we need less research but better research.”
In 2009, a Lancet series on avoidable waste in research estimated not much had changed: 85% of research spending currently goes to waste.
Bad Pharma collated countless problems with the production of evidence: many medical tests and trials are profoundly flawed, evidence is often hidden by drug companies to the detriment of patient care. Add to this is a poor regulatory system then it is increasingly clear we need a radically different system than the current defective system for producing evidence.
John Ionnadis pointed out research is more likely to be false, particularly when effects are small, when many outcomes are presented and when there are greater financial interests. Moreover, “with increasing bias, the chances that a research finding is true diminish considerably”.
An empirical analysis of 102 randomised trials found that half of the efficacy and two thirds of the harm outcomes were incompletely reported, with statistically significant outcomes being more than twice as likely to be reported.
The COMPare Projects analysis of outcome switching in clinical trials of the top five medical journals reports little has changed: on average each trial in the cohort only reported 62% of its specified outcomes.
Effects of reporting bias include overestimation of efficacy and underestimation of safety and is a widespread phenomenon.
A substantial proportion of trials have ghost authorship, which often go undisclosed, and undermine the validity of the results.
Financial and non financial conflicts of interests are widespread amongst academic institutions and researchers and is associated with pro industry conclusions, restrictions on publication and data sharing and “private interests”.
Analysis of 32 Cancer drugs reported 2014 drug costs were on average six times higher than those in 2000: costing on average $11,325 compared to the $1,869 per month in 2000.
86% of 92 Cochrane reviews did not include data from the main harm outcome and the primary harm outcome was inadequately reported in 76% of the 931 included trials in these reviews.
Analysis of 462 medicinal products withdrawn from the market found only 43 (9%) were withdrawn worldwide, and the interval between the first reported adverse reaction and year first withdrawal was a median 6 years.
A systematic review of 39 studies reported no robust studies have evaluated shared decision making strategies, signifying it is difficult to advise which strategy, if any, to adopt when it comes to informing patients in real world practice.
Analysis of 20,000 Medicare patients with a principal diagnosis of heart failure reported only 13–25% met the criteria for 3 of the pivotal RCTs.
A systematic sampling review of the eligibility criteria of 283 RCTs published between 1994 and 2006, in high impact general medical journals, reported common medical conditions led to exclusions in 81% of trials and commonly prescribed medications in 54%.
Amongst 155 RCTs of drugs frequently used by elderly patients, with chronic medical conditions, only three studies exclusively included elderly patients.
The pharmaceutical industry influence has led to shorter review times, fast track reviews and less thorough reviews. User fees mean drug companies are the main client of the US FDA, and as a consequence what drugs are developed and how they are tested, is left up to the pharmaceutical industry and the quality of the regulatory reviews is declining.
From 2009 to 2014 the pharmaceutical industry received fines totalling $13 billion for criminal behaviour and civil infringements – behaviour that has largely gone unnoticed. The three worst offenders were GSK (fine $3 billion) for marketing paxil to children and misleading the FDA; Pfizer ( $2.3 billion) for misbranding Bextra with “the intent to defraud or mislead,” and Johnson & Johnson ($2.2 billion) for illegal promotion of prescription drugs.
Surrogate outcomes are easier to measure the important patient outcomes and their use is on the risle. However, overinterpretation of their effects can lead to misinterpretation of the evidence, often ignoring important harms.
Only a small minority of the trials done are analysed in up-to-date systematic reviews.
“Despite repeated calls to prohibit or limit conflicts of interests among authors and sponsors of clinical guidelines, the problem persists”
Overdiagnosis is common and occurs more frequently with cancer screening, and extends across a range of conditions and currently health professionals are poorly informed.
The cost of clinical drug trials has risen significantly to the point where it is hindering the development of new medicines and preventing trial replication
A significant number of RCTs stop earlier than planned due to apparent benefits that overestimate their true effectiveness. These trials often receive greater media attention and affect clinical practice. Thisis particularly so when the number of events are small