Intended for healthcare professionals

Education And Debate

The New Zealand priority criteria project. Part 1: Overview

BMJ 1997; 314 doi: (Published 11 January 1997) Cite this as: BMJ 1997;314:131
  1. David C Hadorn, manager, special projectsa,
  2. Andrew C Holmes, senior medical advisera
  1. a National Advisory Committee on Health and Disability Ministry of Health Wellington New Zealand
  • Accepted 16 October 1996


New Zealand restructured its health system in 1992 with the aim of achieving greater levels of assessment and accountability in the publicly funded health sector. A committee was established specifically to advise the minister of health on the kinds, and relative priorities, of health services that should be publicly funded. One of its projects has been to develop standardised sets of criteria to assess the extent of benefit expected from elective surgical procedures. These have been developed with the help of professional advisory groups using a modified Delphi technique to reach consensus. So far the committee has developed criteria for cataract surgery, coronary artery bypass grafting, hip and knee replacement, cholecystectomy, and tympanostomy tubes for otitis media with effusion. These criteria incorporate both clinical and social factors. Use of priority criteria to ensure consistency and transparency regarding patients' priority for surgery is required for access to a dedicated NZ$130m (£57m; US$90m) pool of money, created to help eliminate surgical waiting lists and move to booking systems. The criteria will also be used in surgical outcome studies, currently in the planning phase.


In this article we describe a national project, sponsored jointly by New Zealand's National Advisory Committee on Health and Disability and the four regional health authorities, to develop standardised priority assessment criteria for elective surgical procedures. Under the auspices of this project, criteria were developed for cataract extraction, coronary artery bypass graft surgery, hip and knee replacement, cholecystectomy, and tympanostomy tubes for otitis media with effusion. These criteria are used (a) to assess patients' relative priority for surgery, (b) to ensure consistency and transparency in the provision of surgical services across New Zealand, and (c) to provide a basis for describing the kinds of patients who will or will not receive surgery under various possible levels of funding.

New Zealand health reforms

As part of a sweeping overhaul of its economy and social structure, New Zealand implemented major reforms of its healthcare system in 1992 (see box).1 These reforms can be viewed as a response to the imperatives described by Relman in his 1988 editorial in the New England Journal of Medicine announcing the arrival of the era of assessment and accountability in health care.2 Relman called for a “revolution” in how health care is provided and paid for, endorsing a proposal put forth by Elwood in the same journal just a few months earlier.3 Elwood described the problem like this:

Too often, payers, physicians, and health care executives do not share common insights into the life of the patient. We acknowledge that our common interest is the patient, but we represent that interest from such divergent, even conflicting, viewpoints that everyone loses perspective. As a result, the health care system has become an organism guided by misguided choices; it is unstable, confused, and desperately in need of a central nervous system that can help it cope with the complexities of modern medicine.

New Zealand health reforms

  • Fourteen area health boards were replaced with four regional health authorities, which purchase publicly funded health and disability services. The National Advisory Committee on Health and Disability was created to advise the minister of health on the kinds of services to be purchased with public funds-and their priority.

  • The Ministry of Health (formerly Department of Health) is responsible for macro policymaking and funding. Inpatient services are provided predominantly by crown health enterprises (hospitals and affiliated institutions), which are managed as businesses and are state owned.

  • A complete split exists between funding, purchasing, and provision of services.

  • Since this paper was prepared for publication the New Zealand health reforms have themselves been drastically reformed. Future papers in the BMJ will describe these changes.

The New Zealand health reforms represent an effort to provide such a central nervous system. Elwood proposed that the healthcare system should routinely collect detailed clinical information concerning (a) the quantity and kinds of services provided, (b) the numbers and kinds of patients receiving those services, and (c) the outcomes experienced by those patients. Recognition of the need for such assessment data and for better channels of communication constituted a major rationale for the restructuring. At the same time, the contract mechanism was seen as a useful method for ensuring provider accountability.

National health committee

A major component of the legislation under which the healthcare system was restructured was the creation of a National Advisory Committee on Core Health and Disability Support Services, since renamed the National Advisory Committee on Health and Disability-and known as the national health committee. This committee is charged with providing independent advice to the minister of health (independent, in particular, of the Ministry of Health) concerning the “kinds, and relative priorities, of public health services, personal health services, and disability services that should, in the committee's opinion, be publicly funded.”4

Early in its tenure the national health committee came under considerable pressure to develop a relatively simple list of services depicting what was in or out of the “core” of services that would be publicly funded. From the outset, however, the committee has taken a different approach. It has preferred to define eligibility for services in terms of clinical practice guidelines or explicit assessment criteria which depict the circumstances under which patients are likely to derive substantial health benefit from those services, bearing in mind competing claims on resources. Thus, for example, patients could reasonably expect to receive coronary bypass graft surgery at the taxpayer's expense if (and only if) their clinical circumstances were commensurate with a likelihood of substantial benefit from that procedure.

The waiting list problem

Long waiting lists for elective surgery have been a nagging issue that long predated the formation of the ministry, regional health authorities, and the national health committee. Based on one of its early commissioned reports,5 the national health committee recommended that surgical services should move away from a system of waiting lists and toward a system of specific booking times, so that patients would know (within reasonable limits) when they would receive their operation. In addition, the committee called for greater transparency and consistency in the process used to decide priority for elective surgery.6

The minister, the ministry, and the regional health authorities accepted the national health committee's advice, including the replacement of waiting lists with booking systems. As a step toward realising this goal, the regional health authorities and national health committee cosponsored a national project to put in place the tools needed to assess the extent of patients' overall priority or urgency for surgery. These priority criteria would reflect primarily the benefit expected from surgery. Priority would generally be given to patients with the greatest likely benefit.

Thus, the ethical framework under which the project was conducted was largely utilitarian in nature, with the principal goal being to achieve the maximum possible health gain with the available funds. The national health committee had formally embraced the philosophy of maximising expected benefit in one of its early reports.6

The national priority criteria project

A six member project steering group was constituted, consisting of representatives of the national health committee (DCH and ACH) and the surgical services managers of the four regional health authorities. Ministry of Health officials were briefed regularly but were not members of the steering group. The stated objective of the project was:

To develop national criteria for assessing the priority which should be given to patients for medical and surgical procedures.… The national priority criteria will serve the following purposes:

  1. To ensure that the process used to define priority is fair and consistent across New Zealand.

  2. To permit the assessment and comparison of need, case mix, and severity.

  3. To assist the regional health authorities in developing new booking strategies, including target booking times for patients with defined levels of priority.

  4. To permit comparison of waiting times across regional health authorities.

  5. To ensure that social values are integrated into the decision making process in an appropriate and transparent manner.

  6. To provide the framework for the national health committee to define maximum acceptable waiting times for patients with defined levels of priority, as well as core levels of each service.

  7. To make possible national studies on the health outcomes experienced by patients who do or do not receive the services.

The box ‘Project methods’ summarises the approach taken to develop the criteria.

Project methods

  • A summary of the relevant literature was prepared by project staff.

  • Professional advisory groups were constituted for each procedure, consisting of two or three specialists and surgeons from each of the four regions and two general practitioners.

  • A two stage Delphi process preceding each professional advisory group meeting was open to all relevant specialists and surgeons in New Zealand (about 20-30 clinicians participated for each procedure, not counting members of the professional advisory groups).

  • Criteria were selected and initial weights agreed at meetings of the professional advisory groups. The draft criteria were pilot tested and their weights recalibrated based on the results.

Progress to date

Five sets of standardised assessment criteria were developed for elective surgical procedures under the auspices of the project. Numerical scores were assigned to each of the multiple levels of severity on each criterion; relevant scores on each criterion were added together to form a total score. These multiple factor, additive systems are known as linear models. Such models are well known to outperform unaided clinical judgment on a wide variety of diagnostic and predictive tasks.7 8 9

The procedures covered are (in order of development):

  1. Cataract extraction

  2. Coronary artery bypass graft surgery

  3. Hip and knee replacement

  4. Cholecystectomy

  5. Tympanostomy tubes for otitis media with effusion

Table 1 shows the criteria for cataract extraction and table 2 those for hip and knee replacement. All criteria were subject to a pilot study to assess the extent of correspondence between the total priority score and global clinical judgments of urgency. A description of the development of the criteria for coronary artery bypass grafting together with their pilot study are described in part 2 of this article. Additional information on the pilot studies is available on the BMJ's Internet web site (

Table 1

Priority criteria for cataract surgery (maximum score 100)

View this table:
Table 2

Priority criteria for major joint replacement (maximum score 100)

View this table:

Social factors considered in setting priorities

As well as clinical criteria, several social factors were discussed during the course of this project and, to some extent, incorporated within the priority criteria. The most important of these were (a) age, (b) work status, (c) whether patients were caring for dependants or threatened with the loss of their own independence, and (d) time spent on the waiting list.


There was substantial disagreement among project participants about the appropriate role of patients' age in assessing the expected benefit from surgery. From a practical perspective, many participants considered age to be a roughly reliable guide to the overall extent of comorbidity experienced by patients, which in turn affects the extent of benefit that can be expected from surgery. However, others were concerned that, even if this is true on average, use of age as a factor in deciding priority for surgery could result in denying services to many elderly patients who would benefit as much as (or more than) younger patients. In the end, age was incorporated in just one set of criteria: those for coronary artery bypass graft surgery. The rationale for its inclusion here was that this type of surgery has direct implications for life expectancy as well as quality of life, whereas the other surgical procedures directly affect only quality of life. The professional advisory group on coronary artery bypass grafting believed that life prolongation becomes progressively less important for elderly patients compared with the importance of quality of life. Accordingly, the group developed a formula to adjust downward, beginning at age 70, the weights assigned to variables associated with improvements in life expectancy (see BMJ web site for details.)

Threat to independence, care of dependants, ability to work

During the process of identifying the factors currently used by clinicians to make judgments of expected benefit project participants acknowledged that clinicians take into account whether (and to what extent) patients' clinical conditions threaten their ability to work, care for dependants, and live independently. Substantial discussion was held on this topic at each professional advisory group meeting, with clinicians generally agreeing that these factors should be represented as priority criteria. Nevertheless, a certain degree of misgiving was usually noted about incorporating these social factors. To address this issue, the national health committee sponsored two public hearings, one on each major island, specifically devoted to discussing the appropriateness of including these factors in the assessment of urgency and priority for elective surgery. A stratified random sample of the public in each community and patients with the relevant conditions were recruited to provide their perspectives. Clinicians from the local area and members of the professional advisory groups also attended. Although no definitive resolution was achieved, the results of the hearings were regarded by observers from the national health committee and regional health authorities as supporting the inclusion of these factors provided they are given relatively little weight compared to clinical factors.

Time spent on waiting list

The length of time spent waiting for the procedure also proved a contentious and difficult issue. Many clinicians favoured inclusion of such a factor on grounds that the “simple act of waiting” should warrant some consideration. However, this concern was balanced by the fact that, if waiting time were incorporated, the inevitable result would be that in many cases less impaired patients would be operated on before more impaired patients. In the end, “time spent waiting” was excluded from the criteria, mainly because the principal tenet of the criteria is that they reflect the degree of clinical (and social) likely benefit associated with the clinical condition, not time spent waiting.

Minister of health's announcement

On 8 May 1996 the minister of health, Jenny Shipley, announced the creation of a new NZ$130m (£57m; US$90m) fund with the express purpose of reducing waiting times and clearing waiting lists. Access to the funds is contingent on the use of explicit clinical priority criteria, such as, but not limited to, those developed during this project.

Professional and public response is generally positive

Response to the new waiting list initiative has been generally positive. In particular, the response from doctors has been largely one of relief that thousands of patients on waiting lists will now be provided with surgery who would not have received it without these new funds. News coverage has also been generally favourable. The capital's Dominion described the move as another “welcome step toward reducing waiting lists for non-urgent surgery in a responsible way, instead of resorting to the bad old practice of throwing money at a problem and hoping for the best….The new system is designed to ensure that people with the biggest need and greatest potential benefit will have their surgery first, that the same rules apply throughout New Zealand…. All this is light years ahead of rationing surgery by making people wait indefinitely for it, and with marked regional variations.”10

In part 2 of this article we describe in more detail our experience developing, testing, and implementing the priority criteria for coronary artery bypass graft surgery.


We thank the many clinicians who participated in this project without whose support this project could not have been completed successfully.

Funding: National Advisory Committee on Health and Disability and the four regional health authorities.

Conflict of interest: None.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.