Correcting outcome data for case mix in stroke medicineBMJ 1996; 313 doi: http://dx.doi.org/10.1136/bmj.313.7063.1005c (Published 19 October 1996) Cite this as: BMJ 1996;313:1005
Doctors who perform abortions for social reasons are social technicians
- David Barer, Professor of geriatric medicine,
- John Ellul, Research fellow,
- Caroline Watkins, Research coordinator
- Institute for the Health of the Elderly, Newcastle General Hospital, Newcastle upon Tyne NE4 OBE
- Aintree Stroke Unit, Fazakerley Hospital, Liverpool L9 7AL
Structure and process should be audited, rather than outcomes
EDITOR,—The method used by Richard J Davenport and colleagues to correct for differences in case mix before and after a stroke unit was set up could obscure genuine differences in outcomes due to changes in medical care.1 A logistic regression model containing 19 prognostic variables, whose coefficients are derived from the study itself, is almost certain to overfit the data, so that some differences that are due to treatment may be spuriously “explained” by adjustment for case mix. The method may therefore be unfair to the “before and after” comparisons used in clinical audit studies.
Nevertheless, the paper contains a salutary warning about the dangers of non-randomised comparisons, particularly those that might be used to generate league tables of outcomes in different hospitals or units. The corrections for case mix used by the authors apply only to cases of stroke identified prospectively, which may differ considerably from those identified by the routine hospital coding system. We compared 340 cases of acute stroke (International Classification of Diseases, ninth revision, codes 431, 433–4, 436) identified from the hospital information system with those included on a prospective stroke register over 15 months in a teaching hospital in Liverpool. Of the 420 confirmed cases, 278 (66%) were on the hospital information system, but many patients with minor or non-paretic strokes (often misdiagnosed as transient ischaemic attacks) and some of those with rapidly fatal strokes were missed. Sixty two patients identified from the hospital information system had a false positive diagnosis of stroke; many of these patients had been admitted for other reasons, having had strokes previously. Thus the overall case mix was quite different from that reported by the authors.
Even when cases are identified prospectively, comparisons between different units can be hazardous. Not only would few hospitals be able to collect the detailed information on case mix used in Davenport and colleagues' study, but much of it is subject to wide interobserver variability.2 3 For instance, in multicentre comparative studies for the European stroke database project we have found large differences in the proportion of total anterior circulation strokes, which have mostly been due to differences in doctors' willingness to assess key signs (such as visual fields) in drowsy or dysphasic patients.4
Because of these difficulties it is unlikely that valid purchasing decisions can be based on such comparisons among units in the foreseeable future. Randomised controlled trials are needed to establish the relation between the structure or process of care and the outcome, and thereafter it is structure and process rather than outcomes that should be audited.5