Users' guide to detecting misleading claims in clinical research reports

doi:10.1136/bmj.329.7474.1093

BMJ 2004;329:1093-1096 (6 November), doi:10.1136/bmj.329.7474.1093

Education and debate

Users' guide to detecting misleading claims in clinical research reports

Victor M Montori, Assistant Professor¹, Roman Jaeschke, clinical professor of medicine², Holger J Schünemann, associate professor of medicine³, Mohit Bhandari, assistant professor of medicine³, Jan L Brozek, lecturer in medicine⁴, P J Devereaux, assistant professor of medicine³, Gordon H Guyatt, professor of medicine³

¹ Department of Medicine, Mayo Clinic College of Medicine, Rochester MN, USA, ² Department of Medicine, McMaster University, Hamilton ON, Canada, ³ Department of Clinical Epidemiology and Biostatistics, McMaster University, ⁴ Polish Institute for Evidence Based Medicine, Jagiellonian University Medical School, Krakow, Poland

Correspondence to: G H Guyatt guyatt@mcmaster.ca

Plenty of advice is available to help readers identify studieswith weak methods, but would you be able to identify misleadingclaims in a report of a well conducted study?

The first 150 words of the full text of this article appear below.

Introduction

Science is often not objective.¹ Emotional investment in particularideas and personal interest in academic success may lead investigatorsto overemphasise the importance of their findings and the qualityof their work. Even more serious conflicts arise when for-profitorganisations, including pharmaceutical companies, provide fundsfor research and consulting, conduct data management and analyses,and write reports on behalf of the investigators.

Although guides to help recognise methodological weaknessesthat may introduce bias are now widely available,^2
3 these criteriado not protect readers against misleading interpretations ofmethodologically sound studies. In this article, we presenta guide that provides clinicians with tools to defend againstbiased inferences from research studies (box).

Guide to avoid being misled by biased presentation and interpretationof data

Read only the Methods and Results sections; bypassthe Discussion section
Read the abstract reported in evidencebased secondary publications
Beware faulty comparators
Bewarecomposite endpoints
Beware . . . [Full text of this article]

Read methods and results only

-->

Read the abstract reported in pre-appraised resources

Beware faulty comparators

Beware composite end points

Beware small treatment effects

Beware subgroup analyses

Conclusion

CiteULike

Complore

Connotea

Del.icio.us Add to Digg

Digg

StumbleUpon Add to Technorati

Technorati What's this?

Relevant Articles

Incorporating considerations of resources use into grading recommendations: Gordon H Guyatt, Andrew D Oxman, Regina Kunz, Roman Jaeschke, Mark Helfand, Alessandro Liberati, Gunn E Vist, Holger J Schünemann for the GRADE working group
BMJ 2008 336: 1170-1173. [Extract] [Full Text] [PDF]

Waking up from the DREAM of preventing diabetes with drugs: Victor M Montori, William L Isley, and Gordon H Guyatt
BMJ 2007 334: 882-884. [Extract] [Full Text] [PDF]

Trial protocols: time for more than tinkering: James Penston
BMJ 2005 330: 421. [Extract] [Full Text]

Users' guide to detecting misleading claims in research: Misleading claims may be symptom of even more serious flaws: James Penston
BMJ 2005 330: 145. [Extract] [Full Text] [PDF]

Users' guide to detecting misleading claims in research: Discussion section helps redress the balance: S Anuradha
BMJ 2005 330: 145. [Extract] [Full Text]

Users' guide to detecting misleading claims in research: Meta-analyses have led to misleading claims in the past: Peter J Horsey
BMJ 2005 330: 145. [Extract] [Full Text]

Avoid being misled by claims in robust studies
BMJ 2004 329: 0. [Full Text]

This article has been cited by other articles:

Doucet, M, Sismondo, S (2008). Evaluating solutions to sponsorship bias. J. Med. Ethics 34: 627-630 [Abstract] [Full text]
Guyatt, G. H, Oxman, A. D, Kunz, R., Jaeschke, R., Helfand, M., Liberati, A., Vist, G. E, Schunemann, H. J, for the GRADE working group, (2008). Incorporating considerations of resources use into grading recommendations. BMJ 336: 1170-1173 [Full text]
Keeley, P. W., Waterhouse, E. T., Noble, S. I.R. (2007). The evidence base of palliative medicine: is inpatient palliative medicine evidence-based?. Palliat Med 21: 623-627 [Abstract]
Abedin, P., Daniels, J., Khan, K. S (2007). Health technology assessment in obstetrics and gynaecology. Part 2: application in practice. The Obstetrician and Gynaecologist 9: 181-187 [Abstract] [Full text]
Montori, V. M, Isley, W. L, Guyatt, G. H (2007). Waking up from the DREAM of preventing diabetes with drugs. BMJ 334: 882-884 [Full text]
(2005). Hit parade. BMJ 330: 485-485 [Full text]
Penston, J. (2005). Trial protocols: time for more than tinkering. BMJ 330: 421-421 [Full text]
Penston, J. (2005). Users' guide to detecting misleading claims in research: Misleading claims may be symptom of even more serious flaws. BMJ 330: 145-145 [Full text]
Anuradha, S (2005). Users' guide to detecting misleading claims in research: Discussion section helps redress the balance. BMJ 330: 145-145 [Full text]
Horsey, P. J (2005). Users' guide to detecting misleading claims in research: Meta-analyses have led to misleading claims in the past. BMJ 330: 145-145 [Full text]

Rapid Responses:

Read all Rapid Responses

Ethical implications of study design: T.L. James
bmj.com, 5 Nov 2004 [Full text]
Misleading claims - a symptom of even more serious flaws?: James Penston
bmj.com, 8 Nov 2004 [Full text]
Beyond a joke ?: Anna E Livingstone
bmj.com, 10 Nov 2004 [Full text]
Misleading claims about albumin: Mahlon M. Wilkes, et al.
bmj.com, 10 Nov 2004 [Full text]
Re: misleading claims about albumin.: Richard G Fiddian-Green
bmj.com, 11 Nov 2004 [Full text]
Meta analyses and misleading claims: Peter J Horsey
bmj.com, 13 Nov 2004 [Full text]
Is it always misleading?: S Anuradha
bmj.com, 16 Nov 2004 [Full text]
Re: Meta analyses and misleading claims about HRT mortality: Ellen C G Grant
bmj.com, 17 Nov 2004 [Full text]
Albumin, science and the money trough: Ian Roberts
bmj.com, 28 Nov 2004 [Full text]
In response to Rapid Responses about Users' Guides: Victor M. Montori, et al.
bmj.com, 1 Dec 2004 [Full text]
No economic interest?: Mahlon M. Wilkes, et al.
bmj.com, 8 Dec 2004 [Full text]