Intended for healthcare professionals

Rapid response to:

Editorials

Missing clinical trial data: setting the record straight

BMJ 2010; 341 doi: https://doi.org/10.1136/bmj.c5641 (Published 12 October 2010) Cite this as: BMJ 2010;341:c5641

Rapid Response:

Evidence, Good Science and Trust

The editorial by Loder and Godlee [1] highlights the dangers of
selective evidence. However, there is a deeper problem lurking beneath
these troubled waters. Suppressed counter-evidence within trials is only
part of the story. Trials aiming to demonstrate benefits cannot hope to
capture all possible counter-evidence.

It is time to let the scales fall from our eyes and recognise the
inherent weakness in the current approach to evidence based medicine. The
predominant aim in randomised controlled trials is to produce positive
evidence to support a theory. It is always selective. It can never be
complete. Publication bias is virtually inevitable. The publication of
counter-evidence is an unwanted side show. Under the current orthodoxy of
randomised controlled trials, we cannot simply blame commercial pressures
on drugs companies for the emphasis on presenting positive evidence. A
more robust exploration of the limits of the benefits and the extent of
the harms will only result if the emphasis shifts to actively exploring
the realm of applicability [2] of proposed treatment regimes. Until we
know more about when a drug should not be used, we shouldn't fool
ourselves into thinking that the drug has been thoroughly tested.

Loder and Godlee rightly question the hierarchy of evidence: 'Meta-
analyses are generally considered the best form of evidence, but is that a
plausible world view any longer when so many of them are likely to be
missing relevant information?' There is no guaranteed way to do good
science. However, attempting to build a fort of positive evidence to
confirm the benefits of a drug is a folly. Attempting to confirm a theory
while brushing counter-evidence under the carpet has never been a recipe
for good science. We need to free ourselves from the prescriptive
constraints of evidence based medicine and develop more imaginative ways
to test medical interventions.

Loder and Godlee propose that 'Efforts are needed to restore trust in
existing evidence.' However, it would be easy to focus on the issues of
honesty and integrity and ignore the vital issue that it is never possible
to rely on studies whose only goal is to produce positive evidence.

Ultimately, this is about a more general trust. Trust in the medical
profession and trust in the research process guiding medical intervention.
It is time to demonstrate a shared commitment to integrity and the search
for truth if trust in the medical profession is not to continue its
decline.

1. Loder E, Godlee F. Missing clinical trial data: setting the record
straight. BMJ 2010;341:c5641.

2. Hughes I. Serious Nonsense. Book in progress - to be submitted for
publication early in 2011.

Competing interests: No competing interests

15 October 2010
Ian J Hughes
Writer and Independent Consultant
Nottingham, NG7 1AW