BMJ  2003;327:73 (12 July), doi:10.1136/bmj.327.7406.73

Paper

Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective study

¹ Cardiothoracic Unit, Guy's Hospital, London SE1 9RT, ² Clinical Operational Research Unit, University College London, London WC1E 6BT, ³ Society of Cardiothoracic Surgeons of Great Britain and Ireland, London WC2A 3PE

Abstract

Objectives To calculate in-hospital mortality after lobectomy for primary lung cancer in the United Kingdom; to explore the validity of using such data to assess the quality of UK thoracic surgeons; and to investigate the relation between in-hospital mortality and the number of procedures performed by surgeons.

Design Retrospective study.

Setting 36 departments dealing with thoracic surgery in UK hospitals.

Participants 4028 patients who had undergone lobectomy for primary lung cancer by one of 102 surgeons.

Main outcome measures In-hospital mortality in relation to individual surgeons, among all patients, and among each of five groups of patients defined by the number of operations performed by the surgeon.

Results 103 patients (2.6%, 95% confidence interval 2.1% to 3.1%) died after surgery during the same hospital admission. No significant difference was found for in-hospital mortality between the five groups.

Conclusions The number of procedures performed by a thoracic surgeon is not related to in-hospital mortality. Reporting data on in-hospital mortality after lobectomy for primary lung cancer is a poor tool for measuring a surgeon's performance.

Introduction

Since 1998 British cardiothoracic surgeons have had to report their mortality figures for selected "marker" operations. This was in response to publicity surrounding the high mortality from congenital heart surgery at the Bristol Royal Infirmary. Lobectomy for lung cancer was chosen as the marker operation for thoracic surgery because it is the only major thoracic operation performed often enough for meaningful statistical comparisons to be feasible. It is the only general thoracic operation for which data are specific to a surgeon, and data are now available covering two years.

Questions arise as to how such data should be presented and whether the reporting of such data provides a useful tool for assessing the quality of thoracic surgery.¹ We aimed to calculate the mortality from this procedure and in relation to individual surgeons. No other details were available on which to base a risk adjustment. We knew nothing about the patients, such as age or sex, only whether they had left hospital alive or died.

Studies of other surgical procedures have shown a link between how frequently a surgeon performs a procedure and survival of the patient to leave hospital and at five years.^{2 3} The possibility of such a volume effect is of importance in UK thoracic surgery as around half of lobectomies are performed by cardiothoracic surgeons, who perform lobectomy infrequently compared with surgeons who concentrate on lung surgery (table 1). We explored the relation between in-hospital mortality of patients undergoing lobectomy for primary lung cancer and the number of lobectomies performed by the surgeon. From the pooled data it was not possible to identify any individual surgeon, and this was not our purpose. We wanted to highlight some of the pitfalls of interpreting in-hospital mortality in relation to individual surgeons.

View this table: [in this window] [in a new window]   Table 1 Number of lobectomies performed, by specialty of operating surgeon, 1999-2001

 

Methods

Data were collected by a designated member of each thoracic surgical unit (surgeon or data manager) as part of the UK Society of Cardiothoracic Surgeons' annual return and collated centrally.⁴ These returns are not audited externally, but the accuracy of the computer entry is verified by each unit. Operations are categorised under 237 headings and pooled for the unit. Postoperative deaths during an admission for a procedure are recorded, but not the cause of death.

We present data on 4028 patients who had undergone lobectomy for primary lung cancer by one of 102 surgeons reporting in both of the financial years 1999-2000 and 2000-1. Firstly, we calculated the overall mortality from lobectomy with 95% confidence intervals, then we produced a scatter plot of mortality in relation to the number of procedures performed by the surgeon over the two year period. We also plotted absolute mortality over the two years in relation to the number of procedures performed by the surgeon during that time. As a guide, we added three lines to the plot: one corresponding to the overall average in-hospital mortality and two between which 95% of points would be expected to lie if the risk of in-hospital mortality were the same for all patients. We divided the data into fifths according to the number of operations performed by the surgeon over the two years. For each fifth, we calculated the mortality and exact 95% confidence intervals and used a ² test to determine any significant differences in in-hospital mortality between the five groups.²

Results

Of the 4028 patients who had undergone a lobectomy over the two year period, 103 died (mortality 2.6%, exact 95% confidence interval 2.1% to 3.1%) during the same hospital admission. Figures 1 and 2 present the data for each of the 102 surgeons. Figure 1 gives an initial impression of a relation between surgical volume and risk. Most of the data in figure 2² lie within the 95% limits that would be expected if the risk of in-hospital mortality were the same for all patients, regardless of surgeon or other factors. Table 2 presents the mortality among the five groups of patients defined by the number of procedures performed by the surgeon over the two years.

View larger version (20K): [in this window] [in a new window]   Fig 1 In-hospital mortality in relation to number of operations performed by surgeon during 1999-2001

 

View larger version (27K): [in this window] [in a new window]   Fig 2 Number of patient deaths in relation to number of operations performed by surgeon during 1999-2001

 

View this table: [in this window] [in a new window]   Table 2 Data on in-hospital mortality for five groups of patients defined by number of operations performed by surgeon

 

To give a roughly equal number of patients in each group, we allocated the patients of two surgeons that had each performed 64 operations to different groups. Three deaths were related to one of the surgeons and none to the other. At the first analysis the split was made according to the sequence in which the surgeons' cases were listed. We found no significant differences in in-hospital mortality between the five groups (²=1.477, df=4, P=0.83).

Discussion

In-hospital mortality after lobectomy for primary lung cancer is not related to the number of procedures performed by an individual surgeon. The relation between a surgeon's volume and outcome has attracted interest, and it seems reasonable to believe that there could be an effect. Data have been published on 2.5 million operations over five years in the Medicare system.² These data, which were analysed with the model we used, show powerful effects for pancreatic and oesophageal cancer, which also apply to pneumonectomy and lobectomy for lung cancer. Five year survival is also influenced by the number of lung resections for cancer performed by a surgeon.³ This may be a learning curve or practice effect.⁵ It may also reflect better decisions made by teams dealing with lung cancer rather than those whose major focus is cardiac.

The UK data, however, do not show a relation between volume and outcome. Patients in the first group, who were operated on by 49 surgeons with an annual volume of one to 15 lobectomies, compared well with patients in the fifth group who were operated on by just six surgeons doing 47 to 96 lobectomies a year. Across the five groups the mortality varied from 2.0% to 2.9%, with no evidence of relation to volume.

Most of the 102 surgeons we reviewed performed both heart and lung surgery.^{6 7} Lobectomy for lung cancer was thus a small proportion of their work. They typically performed around 150 to 200 cardiac operations a year, so any practice effect for lobectomy (an operation of intermediate technical difficulty in cardiothoracic surgery) is likely to be subsumed under the overall case mix. It is also likely that any volume effect according to setting is lost because the lower volume of lobectomies are most commonly performed within cardiothoracic units, which deal with a large volume of major chest surgery.² In the United Kingdom we need the contribution of "low volume" surgeons (surgeons performing less than 24 lobectomies a year provide 40% of the UK service for this procedure), so our results are reassuring.⁷

Figure 1 gives the impression of a relation between surgical volume and outcome. This can be explained by the eye being drawn to the single high mortality figure of 20%, representing one death out of five cases, and not to the 35 of 59 points along the x axis which represent surgeons performing fewer than 20 lobectomies a year. It is more than likely that at least one surgeon with a low surgical volume would display high mortality purely by chance—for example, if all the patients of all the surgeons faced the same risk of in-hospital death, there would be a 65% chance of at least one of the surgeons who performed five or fewer operations over the two years having a mortality of 20% or more. Similarly deceptive is the way in which the points line up in asymptotic curves. All points representing a single death describe a curve running from 20% to about 2%, as the denominator increases from five to over 50. The same effect is seen as the points for surgeons with two and three deaths line up. The major characteristics of figure 1 are therefore artefacts rather than useful information about the performance of individual surgeons. Figure 2 provides a graphical representation of the data that is less open to misinterpretation. Importantly, the low level of absolute mortality is more obvious than in figure 1.

What is already known on this topic British cardiothoracic surgeons must report their mortality figures for "marker" operations Lobectomy for primary lung cancer is the marker operation for thoracic surgery; half are performed by surgeons who perform the procedure infrequently Studies of other surgical procedures have shown a link between surgical volume and survival to leave hospital What this study adds In-hospital mortality after lobectomy is not related to the number of procedures performed by a surgeon Mortality figures derived from a small number of cases are unreliable and should not be used as the basis of important decisions made by patients Mortality from lobectomy for primary lung cancer is a poor means of measuring surgical performance

So what purpose does the collection of surgeon specific data serve? Annual surgical volumes are too small to form the basis of any statistically valid comparison of one surgeon with another. It is also likely that anaesthesia and postoperative care are as important as surgical competence in determining survival after lobectomy. The more important determinants of survival are patient related and include age, pulmonary function, cardiac status, and other comorbidity. And yet the data collected on lobectomy are not adjusted for risk. The development of risk scoring within thoracic surgery is clearly needed.

Previous pooling of data by the Society of Cardiothoracic Surgeons was based on anonymity. The current system is open to dishonesty and the likelihood of surgeons refusing to operate on high risk cases. For these reasons we believe that the registration of lobectomy data is an inadequate means of measuring surgical competence. Lung cancer is a rapidly fatal disease, and an informed patient might well choose to face a substantial surgical risk for the chance of cure.⁸ A better indicator of quality would be cancer free survival at five years,³ and this would reflect better the performance of the cancer multidisciplinary team, who are jointly responsible for patient selection.

---

The members of the UK Society of Cardiothoracic Surgeons voluntarily contribute their data. The content of this paper was presented to the UK Thoracic Forum on 9 February 2002, and the members present unanimously agreed to these data being presented. The interpretation and opinions are those of the authors.

Contributors: TT is the surgeon responsible to the Society of Cardiothoracic Surgeons of Great Britain and Ireland for the data on thoracic surgery. He prepared the first draft and worked on all editions of the manuscript. He will act as guarantor for the paper. MU prepared the data displays, performed all statistical analyses, and worked on all versions of the manuscript. AB collates the data on thoracic surgery and worked on all versions of the manuscript.

Funding: AB is paid by the Society of Cardiothoracic Surgeons of Great Britain and Ireland to collate the data, but his academic contribution to this work was not funded.

Competing interests: None declared.

References

1. Fine LG, Keogh BE, Cretin S, Orlando M, Gould MM. How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience. BMJ 2003;326: 25-8.[Abstract/Free Full Text]
2. Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002;346: 1128-37.[Abstract/Free Full Text]
3. Bach PB, Cramer LD, Schrag D, Downey RJ, Gelfand SE, Begg CB. The influence of hospital volume on survival after resection for lung cancer. N Engl J Med 2001;345: 181-8.[Abstract/Free Full Text]
4. English TAH, Bailey AR, Dark JF, Williams WG. The UK Cardiac Surgical Register 1977-1982. BMJ 1984;289: 1205-8.[ISI][Medline]
5. Gawande A. The learning curve. New Yorker Jan 28, 2002: 52-61.
6. Keogh BE, Kinsman R. National adult cardiac surgical database report, 2001. London: Society of Cardiothoracic Surgeons of Great Britain and Ireland, 2002.
7. Partridge MR. Thoracic surgery in a crisis. BMJ 2002;324: 376-7.[Free Full Text]
8. Treasure T. Whose lung is it anyway? Thorax 2002;57: 3-4.[Free Full Text]

(Accepted April 17, 2003)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Article

Lobectomy mortality is a poor measure of surgical performance
BMJ 2003 327: 0. [Full Text] [PDF]

This article has been cited by other articles:

• Cassivi, S. D., Allen, M. S., Vanderwaerdt, G. D., Ewoldt, L. L., Cordes, M. E., Wigle, D. A., Nichols, F. C., Pairolero, P. C., Deschamps, C. (2008). Patient-Centered Quality Indicators for Pulmonary Resection. Ann. Thorac. Surg. 86: 927-932 [Abstract] [Full text]  
• Sundaresan, S., Langer, B., Oliver, T., Schwartz, F., Brouwers, M., Stern, H., Expert Panel on Thoracic Surgical Oncology, (2007). Standards for Thoracic Surgical Oncology in a Single-Payer Healthcare System. Ann. Thorac. Surg. 84: 693-701 [Abstract] [Full text]  
• McCabe, J. E, Jibawi, A., Javle, P. M (2007). Radical cystectomy: defining the threshold for a surgeon to achieve optimum outcomes. Postgrad. Med. J. 83: 556-560 [Abstract] [Full text]  
• Treasure, T, Utley, M (2006). Survival after resection for primary lung cancer.. Thorax 61: 649-650 [Full text]  
• Barlesi, F., Boyer, L., Doddoli, C., Antoniotti, S., Thomas, P., Auquier, P. (2005). The Place of Patient Satisfaction in Quality Assessment of Lung Cancer Thoracic Surgery. Chest 128: 3475-3481 [Abstract] [Full text]  
• Berrisford, R., Brunelli, A., Rocco, G., Treasure, T., Utley, M., On behalf of the Audit and guidelines committee of, (2005). The European Thoracic Surgery Database project: modelling the risk of in-hospital death following lung resection. Eur. J. Cardiothorac. Surg. 28: 306-311 [Abstract] [Full text]  

Rapid Responses:

Read all Rapid Responses

Challenges no surgeon would refuse

Richard G Fiddian-Green
bmj.com, 11 Jul 2003 [Full text]

Complexity of case load

James C Hurley
bmj.com, 14 Jul 2003 [Full text]

Preventable or Predicted lobectomy mortality: A calculated or down right callous risk?

Joseph F. KHALIL-MARZOUK
bmj.com, 23 Jul 2003 [Full text]

Making the Excellence more infectious.

Mangesh, A Thorat
bmj.com, 2 Aug 2003 [Full text]