Was Rodney Ledward a statistical outlier?:Statistical method may be difficult to apply in clinical practiceBMJ 2005; 330 doi: https://doi.org/10.1136/bmj.330.7505.1448-b (Published 16 June 2005) Cite this as: BMJ 2005;330:1448
- Benjamin J Cowling, senior research assistant (, )
- Anthony J Hedley, chair professor
EDITOR—Harley et al make an interesting proposal for retrospectively detecting outliers.1 However, we think that, as presented, the method may be quite difficult for clinical audit teams to understand and apply in routine practice.
It might be helpful to think in terms of test statistics. For each consultant in each year, the authors used the Mahalanobis distance to help calculate a kind of test statistic, and they compared this with a reference distribution. From table 2 of their paper (see bmj.com), some 16% of the consultants were flagged as outliers each year. If the most unusual 16% of consultants are flagged each year, then we would expect 3% of consultants to be flagged in three or more of five years simply by chance (simple binomial model). In practice, the authors found that 11 of about 100 consultants were flagged as outliers in three or more years, including Ledward.
We also detected a basic error in the method in the second paragraph of the methods subsection (“Stage 2”). Harley et al used as a cut-off point the mean of the square root ofχ2, which is given by the √7 degrees of freedom, which in their study is stated as √;7 = 2.66. However, the expectation (mean) of √x is not generally the same as the square root of the mean of x. A distribution with 7 degrees of freedom actually has mean 2.55 (52nd percentile), not 2.66 (58th percentile) as quoted in the paper. This could have made a practical difference to the results.
Competing interests None declared.