BMJ  2005;330:929 (23 April), doi:10.1136/bmj.38377.675440.8F (published 15 April 2005)

Paper

Was Rodney Ledward a statistical outlier? Retrospective analysis using routine hospital data to identify gynaecologists' performance

¹ Inter-Authority Comparisons and Consultancy, Health Services Management Centre, University of Birmingham, Birmingham B15 2RT, ² Department of Public Health and Epidemiology, University of Birmingham, Birmingham B15 2TT, ³ Department of Primary Care and General Practice, University of Birmingham

Objectives To investigate whether routinely collected data from hospital episode statistics could be used to identify the gynaecologist Rodney Ledward, who was suspended in 1966 and was the subject of the Ritchie inquiry into quality and practice within the NHS.

Design A mixed scanning approach was used to identify seven variables from hospital episode statistics that were likely to be associated with potentially poor performance. A blinded multivariate analysis was undertaken to determine the distance (known as the Mahalanobis distance) in the seven indicator multidimensional space that each consultant was from the average consultant in each year. The change in Mahalanobis distance over time was also investigated by using a mixed effects model.

Setting NHS hospital trusts in two English regions, in the five years from 1991-2 to 1995-6.

Population Gynaecology consultants (n = 143) and their hospital episode statistics data.

Main outcome measure Whether Ledward was a statistical outlier at the 95% level.

Results The proportion of consultants who were outliers in any one year (at the 95% significance level) ranged from 9% to 20%. Ledward appeared as an outlier in three of the five years. Our mixed effects (multi-year) model identified nine high outlier consultants, including Ledward.

Conclusion It was possible to identify Ledward as an outlier by using hospital episode statistics data. Although our method found other outlier consultants, we strongly caution that these outliers should not be overinterpreted as indicative of "poor" performance. Instead, a scientific search for a credible explanation should be undertaken, but this was outside the remit of our study. The set of indicators used means that cancer specialists, for example, are likely to have high values for several indicators, and the approach needs to be refined to deal with case mix variation. Even after allowing for that, the interpretation of outlier status is still as yet unclear. Further prospective evaluation of our method is warranted, but our overall approach may be potentially useful in other settings, especially where performance entails several indicator variables.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

The value of administrative databases

Mohammed A Mohammed and Andrew Stevens
BMJ 2007 334: 1014-1015. [Extract] [Full Text] [PDF]

Was Rodney Ledward a statistical outlier?: Initial investigations of outliers must be chosen carefully

C Kevin Connolly
BMJ 2005 330: 1448. [Extract] [Full Text]

Was Rodney Ledward a statistical outlier?: Statistical method may be difficult to apply in clinical practice

Benjamin J Cowling and Anthony J Hedley
BMJ 2005 330: 1448. [Extract] [Full Text]

Was Rodney Ledward a statistical outlier?: Ledward's managers knew for 10 years that he was a risk

Nigel Dudley
BMJ 2005 330: 1449. [Extract] [Full Text]

Catch me as soon as you can
BMJ 2005 330: 0. [Full Text] [PDF]

How often are adverse events reported in English hospital statistics?

Paul Aylin, Shivani Tanna, Alex Bottle, and Brian Jarman
BMJ 2004 329: 369. [Full Text] [PDF]

An investigation into general practitioners associated with high patient mortality flagged up through the Shipman inquiry: retrospective analysis of routine data

Mohammed A Mohammed, Anthony Rathbone, Paulette Myers, Divya Patel, Helen Onions, and Andrew Stevens
BMJ 2004 328: 1474-1477. [Abstract] [Full Text] [PDF]

What to do about poor clinical performance in clinical trials • Commentary: Of course patients should be told • Commentary: The surgeon is only one factor

Su Mason, Jon Nicholl, Richard Lilford, Heather Goodare, and Tom Treasure
BMJ 2002 324: 419-421. [Extract] [Full Text] [PDF]

Explaining differences in English hospital death rates using routinely collected data

Brian Jarman, Simon Gault, Bernadette Alves, Amy Hider, Susan Dolan, Adrian Cook, Brian Hurwitz, and Lisa I Iezzoni
BMJ 1999 318: 1515-1520. [Abstract] [Full Text] [PDF]

The Wisheart affair: paediatric cardiological services in Bristol, 1990-5

Peter M Dunn
BMJ 1998 317: 1144-1145. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Mohammed, M. A, Stevens, A. (2007). The value of administrative databases. BMJ 334: 1014-1015 [Full text]  
• Dudley, N. (2005). Was Rodney Ledward a statistical outlier?: Ledward's managers knew for 10 years that he was a risk. BMJ 330: 1449-1449 [Full text]  
• Connolly, C K. (2005). Was Rodney Ledward a statistical outlier?: Initial investigations of outliers must be chosen carefully. BMJ 330: 1448-1448 [Full text]  
• Cowling, B. J, Hedley, A. J (2005). Was Rodney Ledward a statistical outlier?: Statistical method may be difficult to apply in clinical practice. BMJ 330: 1448-1448 [Full text]  

Rapid Responses:

Read all Rapid Responses

Surgeons in the firing line

R Justin Davies
bmj.com, 22 Apr 2005 [Full text]

Should one investigate outliers? or outlying centres?

Pantula SRK Sastry
bmj.com, 26 Apr 2005 [Full text]

Ritchie Inquiry

Jan Chalmers
bmj.com, 27 Apr 2005 [Full text]

Statistical outlier?

C Kevin Connolly
bmj.com, 28 Apr 2005 [Full text]

The Mahalanobis Distance

Benjamin J. Cowling, et al.
bmj.com, 29 Apr 2005 [Full text]

What should we do about outliers?

Simon J Caswell
bmj.com, 2 May 2005 [Full text]

Does it work?

Oliver R Dearlove
bmj.com, 14 May 2005 [Full text]