Was Rodney Ledward a statistical outlier? Retrospective analysis using routine hospital data to identify gynaecologists' performanceBMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38377.675440.8F (Published 21 April 2005) Cite this as: BMJ 2005;330:929
Posted as supplied by author.
Robust Multivariate Outlier Detection using the Mahalanobis Distance
We first undertook a robust principal component analysis, for each year (and all years together) which showed that the seven indicator variables were orthogonal (results not shown). We then carried out a robust multivariate outlier detection analysis, based on the computation of a robust Mahalanobis Distance (MD) for each consultant in each year.
The MD is in essence a weighted (by the sample robust variance-covariance matrix) Euclidean distance. The MD is measured from the centroid of the indicator variable-space, so that a consultant with average values for each variable will have a MD of zero, and this represents the origin. Consultants who are furthest away from the centroid, irrespective of direction will have relatively larger MDs. We used the Stahel-Donoho robust multivariate estimator to compute our MDs (Maronna RA, Yohai VJ. The behaviour of the Stahel-Donoho robust multivariate estimator. J Am Stat Assoc 1995;90:330-41).
For each consultant, we derived approximate 95% intervals of uncertainty using simulation. Since each variable is based on its own sample size (n), which varies with each consultant, we attempted to account for this by taking repeated random samples from an underlying binomial distribution (based on n and p, for the six proportion based variables) and a normal distribution (for the mean length of stay variable). Exploratory analyses indicated that these were reasonable underlying sampling distributions. So for each consultant we produced 1000 randomly simulated values for each of seven variables, and then computed a robust MD in the manner described above from these simulated data sets. From these MDs, we used the 0.95 and 0.05 centiles to estimate the approximate 95% intervals of uncertainty around each consultant’s MD.
The Ö MD distance is known to approximately follow a Öχ2 distribution with k degrees of freedom (k being equal to the number of variables, 7 in our case) with mean equal to Ö k. Where a consultant’s approximate 95% intervals of uncertainty were above the mean (Ö k=2.65), we deemed this consultant to be an outlier. In the presentation of results in Figure 1, we chose to express the Ö MD on the loge scale to aid visualization of the plots.
We also investigated the change in MD over the five years, by taking the transformation loge(Ö MD) using a multi-level modeling approach with the repeated measurements (MD) nested within consultants. The reason for transforming the Ö MD to the loge scale was to avoid heteroscedasticity seen on visual examination of the residuals. The log likelihood of the model with Ö MD -857 versus -238 when using the transformed, loge(Ö MD). We used the standardised residual output from this model to identify outliers beyond +/- 2 standard deviations (Figure 2).
- This Week In The BMJ Published: 21 April 2005; BMJ 330 doi:10.1136/bmj.330.7497.0
- Education And DebateWhat to do about poor clinical performance in clinical trialsCommentary: Of course patients should be toldCommentary: The surgeon is only one factorPublished: 16 February 2002; BMJ 324 doi:10.1136/bmj.324.7334.419
- Education And Debate Published: 24 October 1998; BMJ 317 doi:10.1136/bmj.317.7166.1144
- Dr Foster's Case Notes Published: 12 August 2004; BMJ 329 doi:10.1136/bmj.329.7462.369
- Primary CareAn investigation into general practitioners associated with high patient mortality flagged up through the Shipman inquiry: retrospective analysis of routine dataPublished: 17 June 2004; BMJ 328 doi:10.1136/bmj.328.7454.1474
- Paper Published: 05 June 1999; BMJ 318 doi:10.1136/bmj.318.7197.1515
- Editorial Published: 17 May 2007; BMJ 334 doi:10.1136/bmj.39211.453275.80
- LetterWas Rodney Ledward a statistical outlier?:Statistical method may be difficult to apply in clinical practicePublished: 16 June 2005; BMJ 330 doi:10.1136/bmj.330.7505.1448-b
- Letter Published: 16 June 2005; BMJ 330 doi:10.1136/bmj.330.7505.1449
- LetterWas Rodney Ledward a statistical outlier?:Initial investigations of outliers must be chosen carefullyPublished: 16 June 2005; BMJ 330 doi:10.1136/bmj.330.7505.1448-a
- Measles: 142 000 people died in 2018, mostly aged under 5BMJ December 05, 2019, 367 l6830; DOI: https://doi.org/10.1136/bmj.l6830
- Supporting the school climate strike with a public health “work-in”BMJ December 05, 2019, 367 l6766; DOI: https://doi.org/10.1136/bmj.l6766
- Climate emergency: healthcare systems aren’t prepared for risks to health, warns WHOBMJ December 05, 2019, 367 l6876; DOI: https://doi.org/10.1136/bmj.l6876
- Medical memoirs: using patient stories with explicit consentBMJ December 05, 2019, 367 l6713; DOI: https://doi.org/10.1136/bmj.l6713
- Acute paraplegia in a healthy childBMJ December 05, 2019, 367 l6257; DOI: https://doi.org/10.1136/bmj.l6257
- Learning how to make routinely available data useful in guiding regulatory oversight of hospital care
- Outlier Analysis in Orthopaedics: Use of CUSUM: The Scottish Arthroplasty Project: Shouldering the Burden of Improvement
- Thirty-day postoperative mortality after colorectal cancer surgery in England
- Using routine intelligence to target inspection of healthcare providers in England
- Authors' response
- Unacceptable variation in abdominoperineal excision rates for rectal cancer: time to intervene?
- The value of administrative databases
- Was Rodney Ledward a statistical outlier?: Ledward's managers knew for 10 years that he was a risk
- Was Rodney Ledward a statistical outlier?: Initial investigations of outliers must be chosen carefully
- Was Rodney Ledward a statistical outlier?: Statistical method may be difficult to apply in clinical practice