Original articleThe effect of patient selection on comorbidity-adjusted operative mortality risk: Implications for outcomes studies of surgical procedures
Introduction
A growing body of research on the outcomes of surgical procedures has consistently demonstrated variability in the risk of operative death for patients operated on by different surgeons and in different hospitals 1, 2, 3, 4, 5, 6. Possible sources of variation in the risk of operative mortality (OM) observed between surgeons or surgeon groups include random variation, underlying differences in patient risk, and the quality of care provided to patients. A variety of methodologic techniques have been used to account for the effects of random error and preexisting prognostic differences among patients. After adjusting for baseline differences in risk, many studies 1, 2, 3, 4, 5, 6 still demonstrate differences in OM that are much larger than those that might occur by chance alone. The implication of these studies is that the quality of care provided by surgeons or hospitals whose patients had a lower OM rate is better than that provided by those whose patients had a higher OM rate.
This approach to measuring the quality of surgical care has popular appeal. OM is a clean health care outcome that can be measured easily and precisely, and the notion that better quality of care results in fewer operative deaths has considerable face validity. However, baseline differences in illness severity between patients must be handled carefully to obtain valid estimates of the quality of surgical care. Illness severity may be a classic confounder of the relationship between surgeons or hospitals and OM risk because illness severity is a strong determinant of OM 7, 8, and patients with different severity of illness may be unequally distributed among surgeons or hospitals [9]. Although some studies on the outcomes of surgical procedures have used detailed clinical data to stratify patients into different categories of operative risk 10, 11, 12, most such studies rely on administrative data that typically lack the detail of clinically based risk-adjustment methodologies.
Empirical studies have found that risk-adjusted mortality rates derived from administrative data may perform poorly as a measure of the quality of medical care 9, 13, 14. A possible explanation is that administrative data lack the clinical detail to accurately predict operative risk. If only the presence or absence of a comorbidity is used in a risk-adjustment model, there may be residual differences in the risk of operative death between patients with differing severity of the same comorbidity. If surgeons select patients differentially with respect to illness severity within a comorbidity classification (for example, by systematically selecting only patients with mild congestive heart failure instead of all patients with congestive heart failure), a biased estimate of the risk of OM between surgeons may nevertheless occur in studies that attempt to adjust for case-mix. Differences in the apparent risk of operative death will be observed even when there is no difference in the quality of care provided to patients who have surgery.
To better characterize this problem, we developed a model to investigate the effect of occult selection bias on the basis of comorbidity severity on the short-term mortality risk for surgical procedures. Our model addressed the following question: What relative risk of operative death might be observed between two providers or provider groups as a result of occult selection bias, when there are no true underlying differences in the quality of care provided to patients who actually undergo an intervention?
Section snippets
Development of a Model
Assuming that comorbidity severity can be stratified into two or more discrete levels in terms of the risk of operative death, a proportion (p) of all patients fall into the stratum of highest risk. We assumed that clinicians could correctly identify all of the high-risk patients and that a “liberal” or “nonselective” surgeon would operate on all patients, but a “conservative” or “selective” surgeon would only operate on patients at lower risk than those in the high-risk stratum. We also
Discussion
Compared with randomized controlled trials, where inferences of efficacy are made by comparing treatment effects between groups of precisely characterized individuals, outcomes studies must draw conclusions about the usefulness of interventions by comparing heterogeneous groups of individuals who differ with respect to many known and unknown determinants of outcome in addition to the treatment of interest. To minimize the impact of confounding variables on estimates of treatment effects,
Acknowledgments
We are extraordinarily grateful for the suggestions of two anonymous peer reviewers whose thoughtful commentary on previous versions of this manuscript has been invaluable in the development of this work. This work was supported in part by the physicians of Ontario through the Physicians' Services Incorporated Foundation. Dr. Bell is funded through a Clinician-Scientist Award from the Canadian Institutes of Health Research and the University of Toronto Department of Medicine.
References (24)
- et al.
Abdominal aortic aneurysm repair in Veterans Affairs medical centers
J Vasc Surg
(1996) - et al.
Effect of hospital volume on in-hospital mortality with pancreaticoduodenectomy
Surgery
(1999) - et al.
Complex gastrointestinal surgeryimpact of provider experience on clinical and economic outcomes
J Am Coll Surg
(1999) - et al.
Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical careresults of the National Veterans Affairs Surgical Risk Study
J Am Coll Surg
(1997) - et al.
Risk adjustment of the postoperative morbidity rate for the comparative assessment of the quality of surgical careresults of the National Veterans Affairs Surgical Risk Study
J Am Coll Surg
(1997) - et al.
Experience in the United States with intact abdominal aortic aneurysm repair
J Vasc Surg
(2001) - et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
J Clin Epidemiol
(1992) - et al.
Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data
J Clin Epidemiol
(1996) - et al.
A new method of classifying prognostic comorbidity in longitudinal studiesdevelopment and validation
J Chron Dis
(1987) - et al.
Should operations be regionalized? The empirical relation between surgical volume and mortality
N Engl J Med
(1979)