Intended for healthcare professionals

Rapid response to:

Education And Debate

Users' guide to detecting misleading claims in clinical research reports

BMJ 2004; 329 doi: https://doi.org/10.1136/bmj.329.7474.1093 (Published 04 November 2004) Cite this as: BMJ 2004;329:1093

Rapid Response:

Misleading claims - a symptom of even more serious flaws?

Sir,

Montori et al. draw attention to misleading claims in published
clinical trials. [1] In this regard, the authors’ views are to be
welcomed. Their examples of this practice, as well as those documented
elsewhere, [2,3] suggest that it is not uncommon. Much of their advice –
including the encouragement to use independently produced abstracts,
together with the cautions concerning faulty comparators, composite
endpoints, small treatment effects and subgroup analyses - seem to be
entirely reasonable. There are, though, objections to their more radical
proposal.

Montori et al. recommend that only the methods and results sections
should be read, while the remainder of the paper – and, in particular, the
discussion – should be ignored. Yet, anyone capable of critically
appraising a clinical trial solely on the basis of the methods and results
sections – a challenging task, given the complexity of publications
related to clinical research - is unlikely to be fooled by misleading
claims in the discussion.

The advice to omit the discussion section, however, raises further
problems. The title of the paper by Montori et al. is: “Users’ guide to
detecting misleading claims in clinical research reports”. [1] But how are
these misleading claims to be identified if they are present in the
discussion and yet this section is to be omitted? There appears to be some
confusion. But, more importantly, if the parts of the paper containing the
misleading claims were to be omitted, then the discerning reader would
lose the opportunity of witnessing the conflict between the results of the
study and the unwarranted conclusions. This has implications beyond the
recognition of unjustifiable claims. If the researchers involved in a
clinical trial are willing to disseminate misleading claims, then we
should surely question their integrity. But if their integrity is under
suspicion, then all aspects of the clinical trial – including the contents
of the methods and results sections – are also under suspicion. Misleading
claims should be identified and then broadcast loudly throughout the
medical research community, for they signal doubts about the rest of the
study.

As a footnote to these remarks, I would add that much of the problem
of misleading claims has its origin in the flawed nature of large-scale
randomised trials. The complex and opaque data, the trivial treatment
effects and the absence of any means to verify the findings are ideal
conditions for those intent on manipulating the results for their own
advantage. There is, of course, a simply remedy. In future, when reading
the latest clinical trial, ignore everything except the number of patients
recruited to the study. If this is large, then go no further – the study
is unlikely to be of any value to your patients. [3]

[1] Montori VM, Jaeschke R, Schunemann HJ, et al. User’s guide to
detecting misleading claims in clinical research reports. BMJ
2004;329;1093-6.

[2] Wennberg R & Zimmermann C. The PROGRESS trial three years later:
time for a balanced report of effectiveness. BMJ 2004;329;968-70.

[3] Penston J. Ficition and Fantasy in Medical Research: the Large-Scale
Randomised Trial. The London Press. London, 2003. Chap V.

Competing interests:
None declared

Competing interests: No competing interests

08 November 2004
James Penston
Consultant Physician/Gastroenterologist
Scunthorpe General Hospital, Cliff Gardens, Scunthorpe, North Lincolnshire, DN15 7BH