Papers

Influence of hospital and clinician workload on survival from colorectal cancer: cohort studyCommentary: How experienced should a colorectal surgeon be?

BMJ 1999; 318 doi: https://doi.org/10.1136/bmj.318.7195.1381 (Published 22 May 1999) Cite this as: BMJ 1999;318:1381

Abstract

Objective: To determine whether clinician or hospital caseload affects mortality from colorectal cancer.

Design: Cohort study of cases ascertained between 1990 and 1994 by a region-wide colorectal cancer register.

Outcome measures: Mortality within a median follow up period of 54 months after diagnosis.

Results: Of the 3217 new patients registered over the period, 1512 (48%) died before 31 December 1996. Strong predictors of survival both in a logistic regression (fixed follow up) and in a Cox's proportional hazards model (variable follow up) were Duke's stage, the degree of tumour differentiation, whether the liver was deemed clear of cancer by the surgeon at operation, and the type of intervention (elective or emergency and curative or palliative intent). In a multilevel model, surgeon's caseload had no significant effect on mortality at 2 years. Hospital workload, however, had a significant impact on survival. The odds ratio for death within 2 years for cases managed in a hospital with a caseload of between 33 and 46 cases per year, 47 and 54 cases per year, and 55 cases per year (compared to one with ≤23 cases per year) were respectively 1.48 (95% confidence interval 1.03 to 2.13), 1.52 (1.08 to 2.13), and 1.18 (0.83 to 1.68).

Conclusions: There was no detectable caseload effect for surgeons managing colorectal cancer, but survival of patients treated in hospitals with caseloads above 33 cases per year was slightly worse than for those treated in hospitals with fewer caseloads. Imprecise measurement of clinician specific “events rates” and the lack of routinely collected case mix data present major challenges for clinical audit and governance in the years ahead.

Influence of hospital and clinician workload on survival from colorectal cancer: cohort study

  1. F Kee, professor (f.kee{at}qub.ac.uk)a,
  2. R H Wilson, specialist registrarb,
  3. C Harper, specialist registrar in public health medicinec,
  4. C C Patterson, senior lecturer in medical statisticsa,
  5. K McCallion, specialist registrard,
  6. R F Houston, consultantb,
  7. R J Moorehead, consultant surgeone,
  8. J M Sloan, readerf,
  9. B J Rowlands, professorg
  1. aDepartment of Epidemiology and Public Health, Queen's University of Belfast, Belfast BT12 6BJ
  2. bNorthern Ireland Centre for Clinical Oncology, Belvoir Park Hospital, Belfast BT8 8JR
  3. cNorthern Ireland Cancer Register, Belvoir Park Hospital
  4. dDepartment of Surgery, Queen's University of Belfast, Belfast BT12 6BJ
  5. eUlster, North Down and Ards Health and Social Services Trust, Dundonald, Belfast BT16 1RH
  6. fDepartment of Pathology, Queen's University of Belfast, Belfast BT12 6BJ
  7. gDepartment of Surgery, Queens Medical Centre, Nottingham NG7 2UH
  8. Department of Surgery, Royal Liverpool University Hospital, Liverpool L69 3GA
  1. Correspondence to: Professor Kee
  • Accepted 21 January 1999

Abstract

Objective: To determine whether clinician or hospital caseload affects mortality from colorectal cancer.

Design: Cohort study of cases ascertained between 1990 and 1994 by a region-wide colorectal cancer register.

Outcome measures: Mortality within a median follow up period of 54 months after diagnosis.

Results: Of the 3217 new patients registered over the period, 1512 (48%) died before 31 December 1996. Strong predictors of survival both in a logistic regression (fixed follow up) and in a Cox's proportional hazards model (variable follow up) were Duke's stage, the degree of tumour differentiation, whether the liver was deemed clear of cancer by the surgeon at operation, and the type of intervention (elective or emergency and curative or palliative intent). In a multilevel model, surgeon's caseload had no significant effect on mortality at 2 years. Hospital workload, however, had a significant impact on survival. The odds ratio for death within 2 years for cases managed in a hospital with a caseload of between 33 and 46 cases per year, 47 and 54 cases per year, and 55 cases per year (compared to one with ≤23 cases per year) were respectively 1.48 (95% confidence interval 1.03 to 2.13), 1.52 (1.08 to 2.13), and 1.18 (0.83 to 1.68).

Conclusions: There was no detectable caseload effect for surgeons managing colorectal cancer, but survival of patients treated in hospitals with caseloads above 33 cases per year was slightly worse than for those treated in hospitals with fewer caseloads. Imprecise measurement of clinician specific “events rates” and the lack of routinely collected case mix data present major challenges for clinical audit and governance in the years ahead.

messages

  • Various benefits have been described for multidisciplinary cancer care, but the precise relation between a surgeon's or hospital's caseload and the outcome for the patient is not known

  • Any investigation of a caseload effect at the hospital or practitioner level has to simultaneously account for each factor and adjust adequately for case mix

  • Surgeon had no significant effect on caseload, but patients treated in hospitals with low caseloads (<33 cases per year) had a slightly better survival at 2 years than those treated in hospitals with a higher caseload

  • Defining surgical expertise in terms of volume of activity may be a misdirected and imprecise yardstick for the quality of cancer care; other aspects of the organisation of services may be far more important

Introduction

In 1995, the then chief medical officer, Sir Kenneth Calman, made recommendations for improving cancer care in England and Wales.1 Care was to be focused in a network of cancer units and centres that would have sufficient workload to permit appropriate subspecialisation and the development of expertise.

Those charged with implementing this recommendation have been exercised by the putative association between patient outcome and the number of procedures undertaken, whether by surgeon or by hospital. The Royal College of Surgeons has stated only that all patients requiring surgical management should be treated by surgeons with appropriate training and experience.2

The Clinical Outcomes Group recently stated that the quality of colorectal surgery can have a strong influence on patient survival.3 The Centre for Reviews and Dissemination further advised that concentrating surgery in the hands of those with better results could improve survival.4 Regardless of the reason, surgeon variability may influence outcome by as much as 20%.5 Evidence on the subject is inconclusive for several reasons, but “if there is no relationship, then the whole process of training and accreditation is called into question.”6

Detecting a relation often depends on the range over which observations are made, and commissioners of care may reasonably ask whether there is any relation locally between volume and outcome. Equally importantly, it should be asked whether surgical audit can reliably distinguish the “good achievers” from the rest and thus influence the shape of future services.

Our study attempts to address these issues, in particular whether patient mortality is related to the number of cases for which a surgeon or hospital is responsible.

Subjects and methods

Colorectal cancer register —The Northern Ireland colorectal cancer register was established in 1990 with the support of a local cancer charity. Although it produces an independent annual report, it now works more closely with the Northern Ireland cancer register. Clinical and pathological details of colorectal cancers diagnosed in Northern Ireland residents are collected from all hospitals and pathology laboratories serving the provinces (1.6 million population). Although we have previously reported incidence data, 7 8 this report is based on follow up (to the end of 1996) of the first 5 year cohort diagnosed with colorectal cancer between 1990 and 1994.

Subjects —Data recorded on each patient included the age, sex, and address of the patient, the site and Duke's stage of the tumour, the nature of surgery undertaken (emergency or elective and curative intent or palliative intent), and the consultant surgeon and hospital in charge of the case. The address of the patient was mapped to the electoral ward. The province's 566 wards were divided into fifths on the basis of their Townsend score, allowing each to be categorised according to a measure of material deprivation.

Follow up —Each year a follow up questionnaire is dispatched to the consultant in charge and cause of death is ascertained from the registrar general for all deceased patients. Our report deals with follow up status of patients at 31 December 1996. The median follow up period for our cohort was 53.8 months.

Statistical methods

We first explored simple univariate predictors of survival at 2 years using a χ2 test for each variable. We then identified independent predictors by using logistic regression. Surgery over the 5 year period was undertaken in 19 different hospitals and by 71 different surgeons. The surgeons and the hospitals were ranked according to the annual number of cases for which they were responsible, and the distribution was subdivided into fifths. Thirteen surgeons each managed fewer than 10 cases over the 5 year period, and we aggregated them and their combined total cases59 into one group. Using data from the medical register, we also described each surgeon according to the number of years they had been a medical practitioner.

Table 1

Demographic and case mix characteristics of cases 1990-4

View this table:

We used multilevel logistic regression modelling911 of the survival status at 2 years after surgery to allow for possible clustering among cases within surgeons and among surgeons within hospitals. In constructing this three level hierarchy, we assigned patients to the hospitals in which their surgeon was mainly based.

Table 2

Univariate predictors of survival at 2 years

View this table:

Although we have presented findings using the logistic regression model for fixed follow up at 2 years, the results were broadly comparable to those when Cox's proportional hazards models analysis for the entire follow up period was used.

For illustrative purposes, we split the data into two periods, representing performance for the first 3 years and the last 2 years. The performance of each surgeon (according to their patients' adjusted mortality), was assessed in the first of these periods by comparing it with an (arbitrarily defined) acceptable mortality. We then determined which surgeons achieved or fell below this acceptable standard in the second period.

Results

Of the 3217 new patients registered for the period, 82 (2.5%)either had a biopsy only or there was insufficient information to determine the procedure, and these patients were excluded from the survival analysis.

Before 31 December 1996, 1512 patients (48.2%) had died; table 1 shows their characteristics. In 340 cases (10.8%) inadequate clinical information was retrieved for Duke's stage to be assigned, and in 183 cases (5.8%) insufficient address information precluded postcoding (and Townsend scoring).

In univariate analysis, significant predictors of survival at 2 years were age, Duke's stage, the degree of tumour differentiation, whether the liver was deemed clear by the surgeon, and the type of surgery (table 2). As we observed no significant effect on survival for tumour site, we combined the sites for further analysis.

After adjustment for case mix in a single level logistic regression analysis, there was only weak evidence of surgeon specific heterogeneity in patient survival (χ2=76.1; df=58; P=0.06). Likewise, there was only weak evidence of significant hospital specific heterogeneity in patient outcome. (χ2=27.1; df=18; P=0.08). We found no detectable effect of consultant workload or “experience” on outcome. Table 3 shows the relative odds of death at 2 years derived from the standard and multilevel logistic regression models and the relative hazards from a Cox's proportional hazards model. The base model contains the biological predictors before we simultaneously forced in the hospital and clinician variables.

In both the standard and multilevel models, the hospitals with higher caseloads—particularly those dealing with between 33 and 54 cases per year—had a slightly worse survival.

Using the entire 5 year cohort of patients, we ranked the performance of the surgeons firstly on the basis of unadjusted 2 year survival and then on survival adjusted for all the significant case mix variables in the logistic model (figure). Using an excess mortality of about 15% (roughly equivalent to the 75th centile of the surgeon's distribution), we compared the ranking of surgeons over the first 3 years with that over the last 2 years. Fourteen surgeons would have failed the test if a health authority had designated its specialist colorectal cancer surgeons by selecting those who achieved a mortality within the acceptable limits for patients treated between 1990 and 1993. However, over the next 2 years, six of them would have become a “safe” pair of hands and 11 others would no longer satisfy the criterion.

Figure1

Consultants ranked (unadjusted, or adjusted for case mix) by mortality of patients after 2 years

Table 3

Odds ratios and hazard ratios in multivariate model of survival

View this table:

Discussion

We have shown that mortality within 3-4 years of diagnosis of colorectal cancer is not associated with annual caseload for surgeons in Northern Ireland dealing with up to 34 cases per year. Only 44% of the cases in this region were managed by surgeons dealing with 15 or more cases per year. This probably differs little from elsewhere in the United Kingdom.

The register we used ascertains its cases mostly through the pathology laboratories, so even if cases given only a clinical diagnosis were missed, their exclusion would not have affected the conclusions for surgical performance.

Other studies have investigated intermediate outcomes,12 and although we have not directly studied the surgeons' technical craft, we speculated whether for patients with similar Duke's stage, the type of surgery differed among surgeons with high and low workloads. Indeed, there was no difference in the proportions of curative versus palliative intent or elective versus emergency procedures. The proportion of cases with insufficient data for staging was broadly comparable across workload categories and we do not think this has biased our conclusions. Imprecise case mix adjustment, particularly if it was differential, may have biased the expected mortality. The collection of good quality information by surgeons is thus an indirect measure of outcome quality.

Although McArdle and Hole showed some significant difference between surgeon variation in patients' survival,12 they did not specifically investigate volume effects. Initial analyses from the Trent and Wales audits given in the report of the Clinical Outcomes Group show no case mix adjusted survival advantage (at 2 years) for patients treated by self reported specialists. Differences have been shown, for example, between survival after treatment in teaching and non-teaching hospitals,13 but whether they originate from selection bias or workload is not known.

Significant effects of operator workload have been shown in substantial literature from the United States, but most of these studies have focused on colectomy rather than on colorectal cancer.1416 The necessity to adjust simultaneously for effects of both hospital and operator specific workload is increasingly acknowledged. Poor outcomes in a specific hospital may be the result of the good outcomes of one expert surgeon being swamped by the poor results of several “occasional” surgeons.

Present policy suggests that we use audit to identify the “safe pair of hands.” The practice of surgeons in some districts has already been curtailed on the basis of samples of less than a year's work.17 While audit can identify a learning curve for surgical performance,18 the definition of an expert is rather fragile. If we assumed that the minimum clinically important attrition of survival for any given surgeon was 10%, then at least 150 cases would be required for this to be reliably detected, with 80% power and an α error (one sided) of 0.05.19 The particular outcome event and how its risk varies over time has a bearing on the type of case mix adjustment required.20 Clearly, audit focusing on the relation between volume and survival will, in most instances, be misplaced.

We can offer no explanation for the slightly worse outcome in the hospitals with medium volume workload. The relation is not a simple linear one. In a preliminary analysis we found it sensitive to the number of categories used for grouping. The grade of the operator was unknown to us but under emerging arrangements for clinical governance we think it justifiable to assign outcomes to the consultant in charge. We do not regard the level of findings from our hospital as conclusive, and they may reflect inadequate adjustment for unspecified case mix variables.

Whatever is responsible for the heterogeneity in outcome, such variation is of importance in multicentre trials. The potential benefits of treatments being studied may be swamped by such background variability.21 For example, preoperative radiotherapy has been shown to reduce local recurrence by 20% in university hospitals and by 60% in non-university hospitals.22

Several lessons emerge from our study. Firstly, making survival data available to wider audiences to rank the performance of surgeons may be unwise. It will often not identify the real problems. Even for breast cancer the differences in survival of patients between surgeons with high caseload and low caseload is not related to the type of surgery.23 Rather, the outcomes from colorectal cancer may depend on the clinical organisation of the service, the specialist surgeon being one trained to work effectively in a team and committed to ongoing quality assurance. Bringing together teams of surgeons, oncologists, pathologists, and palliative care specialists inevitably requires a commissioner to consider issues of economy of scale.

For the time being we must recognise that the volume-outcome debate may be a diversion. The rationale for providing multidisciplinary care and the need to enhance the power of clinical trials already provide compelling arguments for centralisation of cancer care.

Acknowledgments

Contributors: FK and RHW formulated the study hypotheses and along with KMcC prepared the first draft of the manuscript. RW oversaw core data collection and collation. CH was responsible for collating survival data. CCP performed all the statistical analyses. JMS, RJM, RFH, and BJR initiated the Northern Ireland colorectal cancer register, and all discussed and edited the final manuscript. FK and RHW will act as guarantors for the paper.

Funding: The Northern Ireland colorectal cancer registry is supported by the Friends of Montgomery House, an independent charity that supports work in the Northern Ireland Centre for Clinical Oncology at Belvoir Park Hospital.

Competing interests: None declared.

References

Commentary: How experienced should a colorectal surgeon be?

  1. Robert Shields, emeritus professor
  1. aDepartment of Epidemiology and Public Health, Queen's University of Belfast, Belfast BT12 6BJ
  2. bNorthern Ireland Centre for Clinical Oncology, Belvoir Park Hospital, Belfast BT8 8JR
  3. cNorthern Ireland Cancer Register, Belvoir Park Hospital
  4. dDepartment of Surgery, Queen's University of Belfast, Belfast BT12 6BJ
  5. eUlster, North Down and Ards Health and Social Services Trust, Dundonald, Belfast BT16 1RH
  6. fDepartment of Pathology, Queen's University of Belfast, Belfast BT12 6BJ
  7. gDepartment of Surgery, Queens Medical Centre, Nottingham NG7 2UH
  8. Department of Surgery, Royal Liverpool University Hospital, Liverpool L69 3GA

    A question that a patient with colorectal cancer will naturally ask is, “Will my chances of being alive in two years' time be enhanced by my choice of surgeon?” Intuitively we feel that this is so, and that the patients of a more experienced surgeon will have less chance of dying or having complications after the operation and greater long term survival. If this is the case, there are wider implications in terms of training in the subspecialty of colorectal surgery and the concentration of treatment in larger centres with high volumes of patients. Such concentration of activity, however, has its downside because these centres may be more remote, making treatment less accessible and increasing the cost for patients and relatives.

    Most, but not all, publications support a positive relation between increased activity (volume) and outcome for colorectal cancer. Not all studies, however, adjust for case mix or severity of illness, but Kee et al's paper attempts to do so.

    The relation between the number of operations that a surgeon performs and the outcome is unlikely to be linear throughout. The threshold for experience to influence outcome (learning curve) must vary according to the operative procedure—lower for colonic cancer and higher for rectal cancer, where the surgical requirements are more demanding, to preserve anal sphincters in low lesions and to reduce the chances of local recurrence by mesorectal excision. The authors found no relation between survival and the site of the tumour.

    The authors are the prisoners of their experimental design because inherent variability is such that there is little chance of showing statistical differences between the groups and subgroups that they analyse. For example, by focusing on mortality at 2 years the authors have combined the 30 day hospital mortality (which will be largely influenced by the type of surgery and postoperative complications—for example, anastomotic leakage) and longer term survival, which will probably be influenced by such factors as the biology of the tumour and the use of adjuvant treatment, of which radiotherapy, in the case of rectal cancer, is important. Combining surgical and non-surgical factors, which could influence mortality at 2 years, introduces too much variability into the analysis. The fact, however, that no statistical difference can be shown does not mean that a real difference does not exist.

    No information is given on how many of the surgeons had received specialist training in colorectal surgery. Indeed, since in each case it was the name of the consultant surgeon in charge of the case that was recorded, we do not know if the operations were performed by consultants, by surgeons in training, or by both. This could be a factor of importance in emergency operations.

    The experience of the surgeons was measured by the number of years on the medical register, surely a rather crude indicator of specialist experience. The number of years as a consultant surgeon would perhaps be a more accurate marker of post-training experience and could presumably be easily obtained in a circumscribed community such as that in Northern Ireland.

    The worst survival in those hospitals that treated 33-54 cases a year defies explanation and must be a statistical oddity. The value of analysing hospital data is questionable. A separate study is required to determine if hospital volume can serve as a surrogate for surgeon volume for achieving good outcomes in colorectal cancer.

    The problem with Kee et al's study is that, with the comparatively small numbers in the various groups and high variability, it becomes statistically impossible to show that surgeons with small caseloads are better or worse than those with higher caseloads. The results must not give the green light to those who wish to defend small volume workload or to encourage the surgeon who does colorectal surgery only occasionally.

    The paper is interesting because it tackles an important subject and invites further study. Much depends on the resolution of the volume-outcome controversy, not only for the individual patient but also for surgical training and the provision of colorectal cancer services.

    View Abstract