Intended for healthcare professionals

Information In Practice

Surgical quality: review of Californian measures

BMJ 2004; 328 doi: (Published 15 January 2004) Cite this as: BMJ 2004;328:152
  1. Michael S Broder (mbroder{at}, vice president1,
  2. Lisa Payne Simon, senior program officer2,
  3. Robert H Brook, director3
  1. 1Zynx Health, Beverly Hills, CA 90212, USA
  2. 2California HealthCare Foundation, San Francisco, CA
  3. 3RAND Health, Santa Monica, CA
  1. Correspondence to: M S Broder
  • Accepted 22 October 2003


Many countries publicly report data on the quality of health care. Because surgical patients often have time to plan their care they are ideal candidates to use such data. We examined the adequacy of publicly reported data about surgical quality in California. We used data specific to California because this state is the most populous in the United States and more surgery is done here than in any other state. We defined surgical procedures as those invasive procedures listed by the National Center for Health Statistics.1

Methods and results

We updated a list of organisations that reported on quality in California in 1998 by telephoning those organisations and searching the internet for new sources of data about quality. We surveyed all organisations and collected data about publicly reported quality measures including procedures covered, risk adjustment method, and new measures planned for release by 2005.

We found 18 organisations that reported 333 measures of healthcare quality. All measures examined quality at the hospital level; none related to quality of individual surgeons, groups of surgeons, or health plans. A total of 32 measures (10%) rated quality for 21 procedures. These 21 procedures accounted for 270 395 (12%) of the 2 381 601 surgical procedures performed in California in 1999 (1).2 The organisations reported structure, process, and outcome measures on 12, 0, and 19 procedures. Several procedures had more than one reported outcome measure. Six of the 10 most common non-obstetric procedures had none (coronary artery stent placement, hysterectomy, cholecystectomy, open reduction or internal fixation of fracture, oophorectomy, and appendectomy). The most often reported outcome was death in hospital and major complication rate, which was reported for 11 different procedures. These procedures comprise 11.6% of the Californian total.

Measures of surgical quality, California, 1999

View this table:

Except for hospital volume, all measures were risk adjusted. For all measures except death rates for solid organ transplants and coronary artery bypass grafting, risk adjustment relied on routinely collected administrative data, using ICD-9-CM (international classification of diseases, 9th revision, clinical modification) discharge codes.

The time lag between collecting data and reporting varied between six and 36 months. Measures other than annual volumes are based on reporting periods of two to three years. As a result, most current measures reflected care delivered between two and five years ago. Except for coronary artery bypass surgery and transplants, all measures relied on data reported under statutory requirements (for example, hospitals receiving Medicare payments must report deaths and complications).

Three new measures are expected by 2005 (related to hip fracture, carotid endarterectomy, and coronary artery bypass grafting). In California by 2005, public reporting of mortality risk adjusted for coronary artery bypass grafting will become mandatory and will include deaths specific to individual surgeons.


We found few data to help a consumer interested in using quality to select a surgeon or hospital, and the existing data had serious shortcomings. We found no data specific to surgeons, that most outcome measures used administrative risk adjustment, and no reporting of process measures or functional assessments—for example, walking after hip replacement. Most common procedures had no associated quality measures at all. For some complex procedures, researchers have found an inverse association between volume (a commonly reported measure) and mortality.3 However, this association may not hold for most surgeries.4 For common surgical procedures, selecting a high volume hospital may not improve outcomes. Only three new measures are expected over the next three years.

For 88% of surgical patients in California, no publicly reported information exists. These patients must rely on indirect measures, such as academic affiliation, to assess quality. Even consumers who have conditions for which data are reported face difficulties in using these limited data.

Without a new major effort to increase both the number of procedures for which quality measures are available and the validity of those measures, most California consumers will not be able to choose surgical providers based on quality. A competitive market cannot exist under these conditions.


  • Contributors MSB collected, analysed, and interpreted the data. RHB had the concept, interpreted the data, and wrote the manuscript. LPS had the concept, collected data, and wrote the manuscript. MSB is guarantor.

  • Funding This work was supported in part by a grant from the California HealthCare Foundation and by the UCLA Building Interdisciplinary Research Careers in Women's Health Program.

  • Conflict of interest None declared.

  • Ethical approval Not needed.


View Abstract