BMJ  2004;328:39-41 (3 January), doi:10.1136/bmj.328.7430.39

Education and debate

Assessing the quality of research

¹ Department of Primary Health Care, University of Oxford, Oxford OX3 7LF, ² Leiden University Medical School, Leiden 9600 RC, Netherlands, ³ James Lind Initiative, Oxford OX2 7LG

Inflexible use of evidence hierarchies confuses practitioners and irritates researchers. So how can we improve the way we assess research?

The first 150 words of the full text of this article appear below.

The widespread use of hierarchies of evidence that grade research studies according to their quality has helped to raise awareness that some forms of evidence are more trustworthy than others. This is clearly desirable. However, the simplifications involved in creating and applying hierarchies have also led to misconceptions and abuses. In particular, criteria designed to guide inferences about the main effects of treatment have been uncritically applied to questions about aetiology, diagnosis, prognosis, or adverse effects. So should we assess evidence the way Michelin guides assess hotels and restaurants? We believe five issues should be considered in any revision or alternative approach to helping practitioners to find reliable answers to important clinical questions.

Different types of question require different types of evidence

Ever since two American social scientists introduced the concept in the early 1960s,¹ hierarchies have been used almost exclusively to determine the effects of interventions. This initial focus was appropriate but has also engendered confusion. Although interventions . . . [Full text of this article]

-->

Systematic reviews of research are always preferred

Level alone should not be used to grade evidence

What to do when systematic reviews are not available

Balanced assessments should draw on a variety of types of research

Conclusions

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies

Erik von Elm, Douglas G Altman, Matthias Egger, Stuart J Pocock, Peter C Gøtzsche, Jan P Vandenbroucke STROBE Initiative
BMJ 2007 335: 806-808. [Extract] [Full Text] [PDF]

Case reports of suspected adverse drug reactions—systematic literature survey of follow-up

Yoon Kong Loke, Deirdre Price, Sheena Derry, and Jeffrey K Aronson
BMJ 2006 332: 335-339. [Abstract] [Full Text] [PDF]

This article has been cited by other articles:

• Heederik, D., van Rooy, F. (2008). Exposure assessment should be integrated in studies on the prevention and management of occupational asthma. Occup. Environ. Med. 65: 149-150 [Full text]  
• Dartnell, J., Hemming, M., Collier, J., Ollenschlaeger, G. (2008). Putting evidence into context: some advice for guideline writers. Evid. Based Nurs. 11: 6-8 [Full text]  
• Elm, E. v., Altman, D. G, Egger, M., Pocock, S. J, Gotzsche, P. C, Vandenbroucke, J. P, STROBE Initiative, (2007). Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335: 806-808 [Full text]  
• von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., Vandenbroucke, J. P., for the STROBE Initiative, (2007). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. ANN INTERN MED 147: 573-577 [Abstract] [Full text]  
• Vandenbroucke, J. P., Elm, E. v., Altman, D. G., Gotzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., for the STROBE initiative, (2007). Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration. ANN INTERN MED 147: W-163-W-194 [Abstract] [Full text]  
• Dartnell, J., Hemming, M., Collier, J., Ollenschlaeger, G. (2007). Putting evidence into context: some advice for guideline writers. Evid. Based Med. 12: 130-132 [Full text]  
• Nicholson, P J (2007). How to undertake a systematic review in an occupational setting. Occup. Environ. Med. 64: 353-358 [Full text]  
• Goldsmith, M. R, Bankhead, C. R, Austoker, J. (2007). Synthesising quantitative and qualitative research in evidence-based patient information. J. Epidemiol. Community Health 61: 262-270 [Abstract] [Full text]  
• Taylor, B. J., Dempster, M., Donnelly, M. (2007). Grading Gems: Appraising the Quality of Research for Social Work and Social Care. Br J Soc Work 37: 335-354 [Abstract] [Full text]  
• Grypdonck, M. H. F. (2006). Qualitative health research in the era of evidence-based practice.. Qual Health Res 16: 1371-1385 [Abstract]  
• Gambrill, E. (2006). Evidence-Based Practice and Policy: Choices Ahead. Research on Social Work Practice 16: 338-357 [Abstract]  
• Littlejohns, P., Chalkidou, K. (2006). Evidence for health policy. Journal of Research in Nursing 11: 110-117  
• Vandenbroucke, J. P. (2006). What is the best evidence for determining harms of medical treatment?. CMAJ 174: 645-646 [Full text]  
• Loke, Y. K., Price, D., Derry, S., Aronson, J. K (2006). Case reports of suspected adverse drug reactions--systematic literature survey of follow-up. BMJ 332: 335-339 [Abstract] [Full text]  
• Ogilvie, D., Egan, M., Hamilton, V., Petticrew, M. (2005). Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go?. J. Epidemiol. Community Health 59: 886-892 [Abstract] [Full text]  
• Materia, E., Baglio, G. (2005). Health, science, and complexity. J. Epidemiol. Community Health 59: 534-535 [Full text]  
• Chou, R., Helfand, M. (2005). Challenges in Systematic Reviews That Assess Treatment Harms. ANN INTERN MED 142: 1090-1099 [Abstract] [Full text]  
• Nicholson, P J, Cullinan, P, Newman Taylor, A J, Burge, P S, Boyle, C (2005). Evidence based guidelines for the prevention, identification, and management of occupational asthma. Occup. Environ. Med. 62: 290-299 [Abstract] [Full text]  
• Newman Taylor, A J, Cullinan, P, Burge, P S, Nicholson, P, Boyle, C (2005). BOHRF guidelines for occupational asthma. Thorax 60: 364-366 [Full text]  
• Guzelian, P. S, Victoroff, M. S, Halmes, N C., James, R. C, Guzelian, C. P (2005). Evidence-based toxicology: a comprehensive framework for causation. Hum Exp Toxicol 24: 161-201 [Abstract]  
• af Geijerstam, J-L, Britton, M (2005). Mild head injury: reliability of early computed tomographic findings in triage for admission. Emerg. Med. J. 22: 103-107 [Abstract] [Full text]  
• Tooth, L., Ware, R., Bain, C., Purdie, D. M., Dobson, A. (2005). Quality of Reporting of Observational Longitudinal Research. Am J Epidemiol 161: 280-288 [Abstract] [Full text]  
• (2004). Assessing the quality of research. BMJ 329: 621-621 [Full text]  
• Vandenbroucke, J. P (2004). Benefits and harms of drug treatments. BMJ 329: 2-3 [Full text]  
• Vandenbroucke, J. P (2004). Commentary: The HRT story: vindication of old epidemiological theory. Int J Epidemiol 33: 456-457 [Full text]  
• Furukawa, T. (2004). Meta-analyses and megatrials: neither is the infallible, universal standard. Evid. Based Ment. Health 7: 34-35 [Full text]  

Rapid Responses:

Read all Rapid Responses

Poor quality research

Ellen C G Grant
bmj.com, 4 Jan 2004 [Full text]

Current hierarchies better than supposed

Robin T Harbour
bmj.com, 8 Jan 2004 [Full text]

Do hierarchies exist?

Oliver R Dearlove
bmj.com, 19 Jan 2004 [Full text]

Eminence based medicine

Grazyna T Adamiak
bmj.com, 2 Feb 2004 [Full text]