Analysis

Clinical importance cannot be ruled out using mean difference alone

BMJ 2015; 351 doi: https://doi.org/10.1136/bmj.h5496 (Published 20 November 2015) Cite this as: BMJ 2015;351:h5496
  1. Christopher Cates, senior clinical research fellow1,
  2. Charlotta Karner, health technology assessment analyst lead2
  1. 1Population Health Research Institute, St George’s, University of London, Cranmer Terrace, London SW17 0RE, UK
  2. 2BMJ Technology Assessment Group
  1. Correspondence to: C Cates ccates{at}sgul.ac.uk
  • Accepted 5 October 2015

Christopher Cates and Charlotta Karner argue that information about individual patient responses should be included in the clinical assessment of treatments

Clinicians and patients who face treatment choices want to know whether the results of randomised trials are clinically important, as well as statistically significant. The result is considered statistically significant if the probability that it is a chance finding is less than one in 20 (P value is less than 0.05). But the P value does not measure the size of the difference between treatment and comparator and therefore is not sufficient to assess clinical importance. The next question is: does the treatment make enough difference in outcome to be worth using?

The minimum important difference (MID) is the least change in a measurement that is judged to matter to the person being treated and is best assessed by patients themselves. For outcomes such as quality of life, which are measured as scores on a continuous scale, the MID is the minimum change in score required for the patient to decide that their treatment has been effective.1 The MID can be used to evaluate the clinical importance of treatments, but there are different ways to do this.2 3

In this article we consider how to assess whether statistically significant results are also clinically important. We show that when the mean difference is statistically significant, clinical importance can be better judged using a combination of the size of the mean difference and the spread of individual responses to treatment.

More than just the mean difference

NICE guideline groups (among others), currently use the GRADE (grading of recommendations assessment, development, and evaluation) approach to evaluate continuous outcomes in clinical trials. The GRADE approach uses the estimate of the mean difference and its 95% confidence interval (CI) to assess whether there is a statistically significant difference …

View Full Text

Sign in

Log in through your institution

Free trial

Register for a free trial to thebmj.com to receive unlimited access to all content on thebmj.com for 14 days.
Sign up for a free trial

Subscribe