Star wars, NHS styleBMJ 2004; 329 doi: https://doi.org/10.1136/bmj.329.7457.107 (Published 08 July 2004) Cite this as: BMJ 2004;329:107
- Richard M Barker, business and development director1,
- Mark S Pearce, senior research associate2,
- Miles Irving, chairman of board1 (email@example.com)
- 1 Newcastle upon Tyne Hospitals NHS Trust, Freeman Hospital, Newcastle upon Tyne NE7 7DN
- 2 School of Clinical Medical Sciences, University of Newcastle upon Tyne, Newcastle upon Tyne
- Correspondence to: M Irving
- Accepted 18 April 2004
Monitoring of performance is generally agreed to be essential for the efficient running of large publicly funded organisations such as the NHS. However, to be effective it must not only be statistically sound but also have the confidence of the organisations to which it applies. A report of the Royal Statistical Society in 2003 noted that “Done badly it [perfomance monitoring] can be very costly and not merely ineffective but harmful and indeed destructive.”1
The performance of NHS trusts is monitored through the star ratings system, which the Department of Health introduced in September 2001. The Commission for Health Improvement was given responsibility for assessment in 2002-3. We investigated the rating system after our trust was downgraded from three stars (the highest rating) in 2001-2 to two stars in 2002-3. Our analysis shows important shortcomings with the method of assessment.
2002-3 star ratings
In 2002-3 the performance of NHS trusts was assessed for nine key targets, together with a balanced scorecard of clinical focus (10 indicators), patient focus (19 indicators), and capacity and capability (seven indicators). Each trust's performance on the key targets was assessed on the basis that they had been “achieved, underachieved, or significantly underachieved.” Trusts were given two penalty points for underachievement of a target and six points for significant underachievement. An overall score was derived by adding scores in key targets and the balanced scorecard. The highest scoring trusts were given three stars. However, trusts that had significantly underachieved in one of the key targets could not be awarded three stars, whatever the overall score.
The assessment showed that our trust had significantly underachieved on one key target, the outpatient …