Hospitals put into special measures in 2013 have cut their mortality: BMJ report misleading
The report in the BMJ of the Dr Foster analysis of the mortality decline in hospitals put under special measures is somewhat misleading. The widely quoted figures of a decline of 9.5% in mortality rates in the 11 trusts put in special measures ‘Keogh’ hospitals compared with 3.3% nationally presumably refers to the figure on page 11 of the Dr Foster report where these numbers can be extracted. The figure does not give units but refers to ‘slopes’ of HSMRs which are presumably per year. Thus rather than ‘mortality’ as general understood, the decline is in quarterly HSMR , so for example the HSMR nationally might have been expected to decline from 100 to 96.7 in the period. In addition the ‘national’ figure is in fact derived from 11 randomly chosen trusts selected with replacement 1000 times. Presumably this what is meant by ‘thousands of randomised samples from other English trusts’ mentioned in the BMJ report. This procedure was done to assess statistical significance and is not really suitable for a summary of the effect of placing hospitals in special measures. The analysis is further complicated by the fact that a ‘broken stick’ model was fitted but the break point was not chosen as the point when the trusts were placed in special measures but rather where the model chose the break point (p10 of Dr Foster’s report) which is not clear in the BMJ report.
Dr Foster have done well to try and allow for ‘regression to the mean’, but it is still a spectre in these analyses especially in view of a steady reduction in HSMR nationally over the last 10 years and they are trying to examine the last year of this period. In my view a more relevant graph to summarise the effect of special measures is that on page 8 of the report. I don’t have the actual data, but using a ruler on the graph, I find that for ‘Keogh’ hospitals for the third quarter of the years 2012, 2013, 2014, the HSMR in the Keogh hospitals were 1.25, 1.15, 1.00 respectively , whereas the national HSMR were 1.15, 1.07, 0.93. Although these are approximate figures, they are hardly convincing evidence, showing drops of 20% and 19% respectively for the Keogh and National hospitals over the two years and the gap narrowing from 10% above national to 7%. Although Dr Foster do mention changes in coding after special measures, it would have been better to give the observed and expected values by quarter separately to convince us that the increased decline in HSMR in the Keogh hospitals was not in fact an increase in expected values.
Competing interests: I lead the development of the SHMI, a potential competitor to the HSMR