We use cookies to improve our service and to tailor our content and advertising to you. More infoClose You can manage your cookie settings via your browser at any time. To learn more about how we use cookies, please see our cookies policyClose
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles.
The report in the BMJ of the Dr Foster analysis of the mortality decline in hospitals put under special measures is somewhat misleading. The widely quoted figures of a decline of 9.5% in mortality rates in the 11 trusts put in special measures ‘Keogh’ hospitals compared with 3.3% nationally presumably refers to the figure on page 11 of the Dr Foster report where these numbers can be extracted. The figure does not give units but refers to ‘slopes’ of HSMRs which are presumably per year. Thus rather than ‘mortality’ as general understood, the decline is in quarterly HSMR , so for example the HSMR nationally might have been expected to decline from 100 to 96.7 in the period. In addition the ‘national’ figure is in fact derived from 11 randomly chosen trusts selected with replacement 1000 times. Presumably this what is meant by ‘thousands of randomised samples from other English trusts’ mentioned in the BMJ report. This procedure was done to assess statistical significance and is not really suitable for a summary of the effect of placing hospitals in special measures. The analysis is further complicated by the fact that a ‘broken stick’ model was fitted but the break point was not chosen as the point when the trusts were placed in special measures but rather where the model chose the break point (p10 of Dr Foster’s report) which is not clear in the BMJ report.
Dr Foster have done well to try and allow for ‘regression to the mean’, but it is still a spectre in these analyses especially in view of a steady reduction in HSMR nationally over the last 10 years and they are trying to examine the last year of this period. In my view a more relevant graph to summarise the effect of special measures is that on page 8 of the report. I don’t have the actual data, but using a ruler on the graph, I find that for ‘Keogh’ hospitals for the third quarter of the years 2012, 2013, 2014, the HSMR in the Keogh hospitals were 1.25, 1.15, 1.00 respectively , whereas the national HSMR were 1.15, 1.07, 0.93. Although these are approximate figures, they are hardly convincing evidence, showing drops of 20% and 19% respectively for the Keogh and National hospitals over the two years and the gap narrowing from 10% above national to 7%. Although Dr Foster do mention changes in coding after special measures, it would have been better to give the observed and expected values by quarter separately to convince us that the increased decline in HSMR in the Keogh hospitals was not in fact an increase in expected values.
Competing interests:
I lead the development of the SHMI, a potential competitor to the HSMR
Hospitals put into special measures in 2013 have cut their mortality: BMJ report misleading
The report in the BMJ of the Dr Foster analysis of the mortality decline in hospitals put under special measures is somewhat misleading. The widely quoted figures of a decline of 9.5% in mortality rates in the 11 trusts put in special measures ‘Keogh’ hospitals compared with 3.3% nationally presumably refers to the figure on page 11 of the Dr Foster report where these numbers can be extracted. The figure does not give units but refers to ‘slopes’ of HSMRs which are presumably per year. Thus rather than ‘mortality’ as general understood, the decline is in quarterly HSMR , so for example the HSMR nationally might have been expected to decline from 100 to 96.7 in the period. In addition the ‘national’ figure is in fact derived from 11 randomly chosen trusts selected with replacement 1000 times. Presumably this what is meant by ‘thousands of randomised samples from other English trusts’ mentioned in the BMJ report. This procedure was done to assess statistical significance and is not really suitable for a summary of the effect of placing hospitals in special measures. The analysis is further complicated by the fact that a ‘broken stick’ model was fitted but the break point was not chosen as the point when the trusts were placed in special measures but rather where the model chose the break point (p10 of Dr Foster’s report) which is not clear in the BMJ report.
Dr Foster have done well to try and allow for ‘regression to the mean’, but it is still a spectre in these analyses especially in view of a steady reduction in HSMR nationally over the last 10 years and they are trying to examine the last year of this period. In my view a more relevant graph to summarise the effect of special measures is that on page 8 of the report. I don’t have the actual data, but using a ruler on the graph, I find that for ‘Keogh’ hospitals for the third quarter of the years 2012, 2013, 2014, the HSMR in the Keogh hospitals were 1.25, 1.15, 1.00 respectively , whereas the national HSMR were 1.15, 1.07, 0.93. Although these are approximate figures, they are hardly convincing evidence, showing drops of 20% and 19% respectively for the Keogh and National hospitals over the two years and the gap narrowing from 10% above national to 7%. Although Dr Foster do mention changes in coding after special measures, it would have been better to give the observed and expected values by quarter separately to convince us that the increased decline in HSMR in the Keogh hospitals was not in fact an increase in expected values.
Competing interests: I lead the development of the SHMI, a potential competitor to the HSMR