Intended for healthcare professionals

Papers

Was Rodney Ledward a statistical outlier? Retrospective analysis using routine hospital data to identify gynaecologists' performance

BMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38377.675440.8F (Published 21 April 2005) Cite this as: BMJ 2005;330:929
  1. Mike Harley, director (M.J.Harley{at}bham.ac.uk)1,
  2. Mohammed A Mohammed, senior research fellow2,
  3. Shakir Hussain, statistician3,
  4. John Yates, professor1,
  5. Abdullah Almasri, visiting statistician3
  1. 1 Inter-Authority Comparisons and Consultancy, Health Services Management Centre, University of Birmingham, Birmingham B15 2RT
  2. 2 Department of Public Health and Epidemiology, University of Birmingham, Birmingham B15 2TT
  3. 3 Department of Primary Care and General Practice, University of Birmingham
  1. Correspondence to: M Harley
  • Accepted 20 January 2005

Abstract

Objectives To investigate whether routinely collected data from hospital episode statistics could be used to identify the gynaecologist Rodney Ledward, who was suspended in 1966 and was the subject of the Ritchie inquiry into quality and practice within the NHS.

Design A mixed scanning approach was used to identify seven variables from hospital episode statistics that were likely to be associated with potentially poor performance. A blinded multivariate analysis was undertaken to determine the distance (known as the Mahalanobis distance) in the seven indicator multidimensional space that each consultant was from the average consultant in each year. The change in Mahalanobis distance over time was also investigated by using a mixed effects model.

Setting NHS hospital trusts in two English regions, in the five years from 1991-2 to 1995-6.

Population Gynaecology consultants (n = 143) and their hospital episode statistics data.

Main outcome measure Whether Ledward was a statistical outlier at the 95% level.

Results The proportion of consultants who were outliers in any one year (at the 95% significance level) ranged from 9% to 20%. Ledward appeared as an outlier in three of the fiveyears. Our mixed effects (multi-year) model identified nine high outlier consultants, including Ledward.

Conclusion It was possible to identify Ledward as an outlier by using hospital episode statistics data. Although our method found other outlier consultants, we strongly caution that these outliers should not be overinterpreted as indicative of “poor” performance. Instead, a scientific search for a credible explanation should be undertaken, but this was outside the remit of our study. The set of indicators used means that cancer specialists, for example, are likely to have high values for several indicators, and the approach needs to be refined to deal with case mix variation. Even after allowing for that, the interpretation of outlier status is still as yet unclear. Further prospective evaluation of our method is warranted, but our overall approach may be potentially useful in other settings, especially where performance entails several indicator variables.

Footnotes

  • Embedded Image An appendix with statistical details is on bmj.com

  • Contributions The project team was headed by MH, who also carried out the preliminary analysis and wrote the first draft of the paper. JY secured funding, undertook literature reviews, and was instrumental in the initial design. SH and MAM undertook the statistical analyses. MAM produced the final draft of the paper. AA, with guidance and support from SH and MAM, undertook the simulation work. All authors contributed to the writing of the final paper. MH is guarantor.

  • Funding The Kings Fund funded the initial stages of this project

  • Competing interests None declared.

  • Accepted 20 January 2005
View Full Text