Original article
The effect of patient selection on comorbidity-adjusted operative mortality risk: Implications for outcomes studies of surgical procedures

https://doi.org/10.1016/S0895-4356(01)00508-XGet rights and content

Abstract

Consumers of outcomes research may assume that risk-adjustment procedures based on patients' comorbid conditions will control for baseline prognostic differences between comparison groups, so that differences in risk-adjusted outcomes represent effects other than those due to differences in comorbidity severity. However, surgeons may differ in their threshold to operate on patients with different intensities of the same comorbidity, which may not be accounted for using commonly employed risk-adjustment methods. We developed a model to explore the effect that selection based on comorbidity severity could have on estimates of the risk-adjusted relative risk (RR) of operative death. Larger effects on the apparent RR of operative death were observed when both the proportion of patients in the high-risk (“selected”) stratum and the relative increase in the risk of death due to being in the high-risk stratum were large. Biased estimates of the risk-adjusted RR of operative death will be observed if surgeons differentially select patients based on comorbidity severity and if differences in comorbidity severity are not captured by the risk-adjustment methodology.

Introduction

A growing body of research on the outcomes of surgical procedures has consistently demonstrated variability in the risk of operative death for patients operated on by different surgeons and in different hospitals 1, 2, 3, 4, 5, 6. Possible sources of variation in the risk of operative mortality (OM) observed between surgeons or surgeon groups include random variation, underlying differences in patient risk, and the quality of care provided to patients. A variety of methodologic techniques have been used to account for the effects of random error and preexisting prognostic differences among patients. After adjusting for baseline differences in risk, many studies 1, 2, 3, 4, 5, 6 still demonstrate differences in OM that are much larger than those that might occur by chance alone. The implication of these studies is that the quality of care provided by surgeons or hospitals whose patients had a lower OM rate is better than that provided by those whose patients had a higher OM rate.

This approach to measuring the quality of surgical care has popular appeal. OM is a clean health care outcome that can be measured easily and precisely, and the notion that better quality of care results in fewer operative deaths has considerable face validity. However, baseline differences in illness severity between patients must be handled carefully to obtain valid estimates of the quality of surgical care. Illness severity may be a classic confounder of the relationship between surgeons or hospitals and OM risk because illness severity is a strong determinant of OM 7, 8, and patients with different severity of illness may be unequally distributed among surgeons or hospitals [9]. Although some studies on the outcomes of surgical procedures have used detailed clinical data to stratify patients into different categories of operative risk 10, 11, 12, most such studies rely on administrative data that typically lack the detail of clinically based risk-adjustment methodologies.

Empirical studies have found that risk-adjusted mortality rates derived from administrative data may perform poorly as a measure of the quality of medical care 9, 13, 14. A possible explanation is that administrative data lack the clinical detail to accurately predict operative risk. If only the presence or absence of a comorbidity is used in a risk-adjustment model, there may be residual differences in the risk of operative death between patients with differing severity of the same comorbidity. If surgeons select patients differentially with respect to illness severity within a comorbidity classification (for example, by systematically selecting only patients with mild congestive heart failure instead of all patients with congestive heart failure), a biased estimate of the risk of OM between surgeons may nevertheless occur in studies that attempt to adjust for case-mix. Differences in the apparent risk of operative death will be observed even when there is no difference in the quality of care provided to patients who have surgery.

To better characterize this problem, we developed a model to investigate the effect of occult selection bias on the basis of comorbidity severity on the short-term mortality risk for surgical procedures. Our model addressed the following question: What relative risk of operative death might be observed between two providers or provider groups as a result of occult selection bias, when there are no true underlying differences in the quality of care provided to patients who actually undergo an intervention?

Section snippets

Development of a Model

Assuming that comorbidity severity can be stratified into two or more discrete levels in terms of the risk of operative death, a proportion (p) of all patients fall into the stratum of highest risk. We assumed that clinicians could correctly identify all of the high-risk patients and that a “liberal” or “nonselective” surgeon would operate on all patients, but a “conservative” or “selective” surgeon would only operate on patients at lower risk than those in the high-risk stratum. We also

Discussion

Compared with randomized controlled trials, where inferences of efficacy are made by comparing treatment effects between groups of precisely characterized individuals, outcomes studies must draw conclusions about the usefulness of interventions by comparing heterogeneous groups of individuals who differ with respect to many known and unknown determinants of outcome in addition to the treatment of interest. To minimize the impact of confounding variables on estimates of treatment effects,

Acknowledgments

We are extraordinarily grateful for the suggestions of two anonymous peer reviewers whose thoughtful commentary on previous versions of this manuscript has been invaluable in the development of this work. This work was supported in part by the physicians of Ontario through the Physicians' Services Incorporated Foundation. Dr. Bell is funded through a Clinician-Scientist Award from the Canadian Institutes of Health Research and the University of Toronto Department of Medicine.

References (24)

  • C.B. Begg et al.

    Impact of hospital volume on operative mortality for major cancer surgery

    JAMA

    (1998)
  • J.A. Sosa et al.

    Importance of hospital volume in the overall management of pancreatic cancer

    Ann Surg

    (1998)
  • Cited by (0)

    View full text