Intended for healthcare professionals

Education And Debate

The new zealand priority criteria project. part 2: coronary artery bypass graft surgery

BMJ 1997; 314 doi: (Published 11 January 1997) Cite this as: BMJ 1997;314:135
  1. David C Hadorna, manager, special projects,
  2. Andrew C Holmesa, senior medical adviser
  1. a National Advisory Committee on Health and Disability Ministry of Health Wellington New Zealand
  1. Correspondence to: Dr Hadorn
  • Accepted 16 October 1996


Priority criteria developed during a national project were used to conduct an audit of all 662 patients on waiting lists for coronary artery bypass surgery in New Zealand during spring 1996. Based on the observed distribution of priority scores, the cost of providing surgery to all patients down to various levels of priority was estimated. Descriptions incorporating life expectancy and quality of life implications of surgery were developed of the kinds of patients who would or would not receive surgery at each of several possible funding levels. Cardiologists and cardiac surgeons agreed that a threshold of 25 points was a reasonable clinical goal but to work with a threshold of 35, which can be sustained with current levels of funding. All agree that the gap between these clinically preferred and currently afforded thresholds is a subject for wider societal dialogue and decision. The ability to measure the size of the gap between clinical desirability and financial sustainability provides a new transparency to the problem of healthcare resource allocation.


This paper discusses several issues arising from the priority criteria project in the context of the criteria developed for coronary artery bypass surgery. We describe the process of developing criteria, including the results of a pilot test, and discuss how the results of a clinical audit of all patients on New Zealand's waiting lists for coronary artery bypass grafting were used to estimate the cost of providing surgery to patients down to each of several possible clinical thresholds. A new government initiative to clear waiting lists is described which requires use of explicit criteria such as those developed in this project. We discuss how cardiologists and cardiac surgeons agreed to accept a specific numerical threshold as indicative of reasonable levels of service provision. Finally, we describe how the criteria were used to identify and describe the kinds of patients who would or would not receive coronary artery bypass surgery at defined levels of public funding.


Development of the criteria for coronary artery bypass grafting as part of the priority criteria project followed similar work reported by clinicians at Greenlane Hospital in Auckland,1 which itself had been motivated by earlier work on waiting lists sponsored by the National Health Committee on Health and Disability.2 3 The results of the Greenlane study, which used a method based on the rating system developed by Naylor and coworkers,4 had called into question the extent to which quantitative measures could capture clinicians' overall judgments of priority and likely benefit. Nevertheless, the Greenlane investigators, all of whom were also members of our professional advisory group on coronary artery bypass grafting, agreed it was important to continue the effort to develop such criteria.

Clinicians' reactions to the project

In general clinicians in New Zealand were very interested in the project and willing to participate despite tight timetables and nominal reimbursement. Almost all clinicians who were nominated by regional health authorities agreed to serve as professional advisory group members, and 20-30 additional clinicians from around the country took time to provide often extensive responses to requests for comments on each procedure. As described in part 1, all relevant specialists and surgeons were invited to provide comments as part of a modified Delphi process.

As might be expected, clinicians had mixed views on the project. The most commonly expressed concern was that the government or the regional health authorities would use the criteria to specify arbitrary numerical cut off points below which surgery would not be funded. In the minds of many clinicians the real problem was that the level of funding for surgical services was inadequate. Developing a priority system in the context of such scarcity would be like “rearranging the deck chairs on the Titanic,” as one commentator put it. Similarly, a member of the professional advisory group for coronary artery bypass grafting wrote, “If the available surgical resource is inadequate, it is not possible to produce a workable numerical system of prioritisation for patients in need of coronary artery surgery.”

Despite these concerns physicians and surgeons from around New Zealand cooperated with this project to a very substantial extent. Two principal reasons for this cooperation were identified. Firstly, clinicians almost universally acknowledged that decisions about urgency and priority were made inconsistently. Often, the “squeaky wheel would get the grease,” and more deserving but uncomplaining patients would be disadvantaged. One cardiologist put it like this:

Manipulation by referring doctors, friends in high places, MP letters, or just persistent nagging, and just slight exaggeration of symptoms, is rampant, and the poor benign patient simply sits on the list and is leap frogged. I support any system which will provide fair, humane, and prognostic order of surgery.

The second major reason for clinicians' cooperation in this project was their wish to develop an objective measure of symptoms and functional status that policymakers could understand. Participating clinicians viewed the development of standardised assessment criteria as having the potential to provide additional, more comprehensible, and possibly dramatic information concerning the extent of “unmet need.”

Development of criteria for coronary artery bypass grafting

The priority criteria for coronary artery bypass grafting were developed by a professional advisory group consisting of seven cardiologists, four cardiac surgeons, one physician, and two general practitioners. These individuals were nominated by the four regional health authorities and by the Royal New Zealand College of General Practitioners. Selection of the criteria followed an iterative modified Delphi consensus process, including consideration of written comments received from an additional 25 cardiologists and surgeons from around New Zealand (see part 1).

As described in part 1, the priority criteria represent the clinical factors–for example, the extent of coronary artery obstruction–that have been shown, or are considered, to be associated with the degree of benefit obtained from the procedure. Numerical scores (or weights) are assigned to each of multiple levels of severity on each criterion; relevant scores on each criterion are then added together to form a total score. This score is considered indicative of the overall degree of benefit expected from surgery (see table 1)).

Table 1

Priority criteria for coronary artery bypass surgery (maximum score 100)

View this table:

In selecting priority criteria for coronary artery bypass grafting the professional advisory group was able to rely to a much greater extent on published outcome studies than were the advisory groups for the other procedures (cataract surgery, hip and knee replacement, cholecystectomy, and grommets for glue ear). For example, a table listing the various possible degrees of coronary artery obstruction was adopted without significant change from a then newly published analysis of 10 years' experience with coronary artery bypass grafting by Duke University investigators.5 The initial weights assigned to these degrees of obstruction were taken directly from this published report, although recalibrated to accommodate the 100 point maximum adopted for each set of criteria.

Weights were assigned to the remaining factors based on additional information in this report and from a meta-analysis of outcomes of coronary artery bypass grafting published during this process.6 It was agreed that these initial weights would be revised as appropriate based on the results of a pilot study. As described in part 1, both a “social factor” and an age adjustment factor were incorporated into the clinical criteria to reflect both common clinical practice and the balance of social values, as gleaned by the national health committee via public meetings and consultation.

Pilot study

A formal pilot study was conducted of each set of criteria. Details of the methods and results of these studies are available on the BMJ's worldwide web site ( We briefly describe the coronary artery bypass grafting criteria pilot study here.

A total of 260 patients were assessed during the study. Of these, 133 patients were evaluated at Greenlane Hospital (Auckland), 119 at Dunedin Hospital, and eight at Waikato Hospital (Hamilton). Although patients were enrolled more or less consecutively during the study period, the sample should be considered a convenience sample.

Total priority scores were calculated for each patient by adding the weights assigned to various factors at the appropriate levels. In addition, physicians were asked to estimate what a “reasonable waiting time” (in days) would be for each patient, considering an “adequately, not infinitely funded service” and “keeping in mind competing claims for resources both within and outside the health sector.” Reasonable waiting time, which was considered indicative of likely benefit, was used as the outcome (dependent) variable in our analyses. Alternative dependent variables could have been used, such as clinicians' global assessment of expected benefit on a scale of 0-100. It is unclear whether the results of our analysis would have differed substantially had an alternative dependent variable been used.

Regression analysis was used to determine the set of criteria weights resulting in the highest degree of correlation between priority scores and clinicians' judgments of reasonable waiting times. Slight modifications were then made in a few weights based on clinical judgment. The final criteria and weights (table 1)) correlated quite closely with estimates of reasonable waiting time, with a statistical test of correspondence (coefficient of variation, or r2) of 0.62 (perfect correlation would score 1.0, no correlation would score 0).

Based on the results of the pilot test we calculated the approximate cost of providing surgery to patients who present for coronary artery bypass grafting in New Zealand on a steady state basis–that is, assuming that a separate (and separately funded) initiative were used to clear the waiting lists (as discussed below). Table 2) shows the estimated cost of providing surgery to patients at or above each of various possible clinical thresholds. On current funding levels we estimate that coronary artery bypass grafting can be provided to patients scoring 35 points or higher.

Table 2

The estimated cost of providing surgery on a steady state basis to patients at or above each of various possible clinical thresholds, highlighting the level of current funding (threshold 35 points)

View this table:

Audit of waiting lists for coronary artery bypass grafting

Following development, testing, and revision of the criteria for coronary artery bypass grafting a clinical audit was conducted of all patients on New Zealand's waiting lists for coronary artery bypass grafting using the revised criteria. A single, experienced, independent nurse reviewer examined the clinical records of all 662 patients on the four regional waiting lists and abstracted from those records the data required for calculating priority scores. Standardised abstraction forms and coding protocols were developed to provide additional assurance of comparability across centres.

The observed distribution of priority scores for patients on waiting lists for coronary artery bypass grafting in New Zealand was roughly normal (fig 1).


Distribution of priority scores for coronary artery bypass grafting among 662 patients waiting for the operation in New Zealand. Numbers of patients are given above each bar

Based on this distribution, we calculated the cost of providing coronary artery bypass grafting surgery to all patients on current public waiting lists at or above specified thresholds of clinical priority (table 3)). These estimates were derived using various assumptions concerning the unit cost of coronary artery bypass grafting and the proportion of patients on lists who would no longer benefit from surgery.

Table 3

Summary of costs for performing coronary artery bypass grafting on 662 patients on New Zealand's waiting lists

View this table:

Describing kinds of patients

The priority criteria used in this project lend themselves to the purpose of providing a “common insight into the life of the patient.”7 In the case of coronary artery bypass grafting, patients were described by reference to five point bands on the scale of clinical priority. Within each band, patients were ordered on each variable and the median values of each variable identified. The collection of median values on all four variables was used to describe the “average patient” within each five point band. Table 4) depicts the results of this process.

Table 4

Median levels of each clinical variable within each 5 point priority score band for coronary artery bypass grafting April 1996

View this table:

For purposes of communicating more directly to politicians, policymakers, and the public a greater degree of descriptive richness was considered necessary. For this reason, the range of priority scores was divided into three levels and the median values of patients within each level identified. Descriptions based on these values were constructed using the operational definitions of angina (table 1)) together with estimates of the likely implications of coronary artery bypass grafting surgery on life expectancy. These estimates were based on an examination of a recent meta-analysis.6 The resulting descriptions are presented in the box. These descriptions were deemed by most observers to be valid and effective descriptions of patient severity with which to communicate to the public and policymakers.

Descriptions of average patient at each of three levels of priority score

  • Patients with a score of 55 or more have considerably reduced quality of life due to chest pain and breathlessness on almost any physical activity and a reduction in life expectancy of perhaps 1-2 years in the absence of surgery.

  • Patients with scores of 35-54 experience much reduced quality of life, mainly through pain on exertion, such as walking one or two blocks, as well as moderately (8-12 months) reduced life expectancy in the absence of surgery.

  • Patients with scores of 25-34 points experience intermittent pain or breathlessness when undertaking such activities as walking or climbing stairs rapidly and experience a modest reduction in life expectancy (4-8 months) in the absence of surgery.

Specification of clinically desirable threshold

As described in part 1, on 8 May 1996 the minister of health announced the creation of a NZ$130m fund to be used for clearing surgical waiting lists and replacing them with booking systems. On that same day the minister also launched a meeting attended by cardiologists, cardiac surgeons, and representatives of the Ministry of Health, national health committee, and regional health authorities.

The results of the audit just described were presented at that meeting. The clinicians accepted the results of the audit and, after discussion, agreed that a clinical threshold of 25 points before considering coronary artery bypass grafting was reasonable given the degree of benefit expected and competing claims on resources. Whether public funding would be sufficient to operate on all patients above this threshold was recognised by all participants to be a separate, societal question. Indeed, at the meeting the minister agreed to be held accountable for any gap between what is clinically desirable and what is financially sustainable, reasoning that appropriate funding levels must take into account competing claims on resources–adjudication of which is ultimately up to society to resolve through democratic processes.

As noted earlier, preliminary estimates indicate that current funding levels will permit surgery to be offered to patients scoring at or above 35 points. As such, there is an apparent 10 point gap between what is clinically preferred and what can be afforded. We believe that the ability to quantify this gap, even if imperfectly, represents a major advantage of the general approach described in this article.

The acceptance by clinicians of a quantitative threshold for surgery, based on priority criteria, represents a key development in the transition within New Zealand from waiting lists to booking systems. Such explicit acceptance by clinicians of the inevitability of limits is vital to the success of any attempt to distribute healthcare dollars more equitably. On balance, we believe the experiences described in this article are an important step towards the goal of a fair, transparent, and evidence based allocation policy.


We thank our professional advisory group for their support and help, especially Dr Trevor Agnew and Mr Richard Bunton; Annmarie Banchy for her excellent work in conducting the national audit of coronary artery bypass graft waiting lists; and Paul O'Connor for performing the statistical analysis.

Funding: National Advisory Committee on Health and Disability and the four regional health authorities.

Conflict of interest: None.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.