Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective studyBMJ 2003; 327 doi: https://doi.org/10.1136/bmj.327.7406.73 (Published 10 July 2003) Cite this as: BMJ 2003;327:73
- Tom Treasure, professor of cardiothoracic surgery ()1,
- Martin Utley, senior research fellow2,
- Alan Bailey, data coordinator3
- 1Cardiothoracic Unit, Guy's Hospital, London SE1 9RT
- 2Clinical Operational Research Unit, University College London, London WC1E 6BT
- 3Society of Cardiothoracic Surgeons of Great Britain and Ireland, London WC2A 3PE
- Correspondence to: T Treasure
- Accepted 17 April 2003
Objectives To calculate in-hospital mortality after lobectomy for primary lung cancer in the United Kingdom; to explore the validity of using such data to assess the quality of UK thoracic surgeons; and to investigate the relation between in-hospital mortality and the number of procedures performed by surgeons.
Design Retrospective study.
Setting 36 departments dealing with thoracic surgery in UK hospitals.
Participants 4028 patients who had undergone lobectomy for primary lung cancer by one of 102 surgeons.
Main outcome measures In-hospital mortality in relation to individual surgeons, among all patients, and among each of five groups of patients defined by the number of operations performed by the surgeon.
Results 103 patients (2.6%, 95% confidence interval 2.1% to 3.1%) died after surgery during the same hospital admission. No significant difference was found for in-hospital mortality between the five groups.
Conclusions The number of procedures performed by a thoracic surgeon is not related to in-hospital mortality. Reporting data on in-hospital mortality after lobectomy for primary lung cancer is a poor tool for measuring a surgeon's performance.
Since 1998 British cardiothoracic surgeons have had to report their mortality figures for selected “marker” operations. This was in response to publicity surrounding the high mortality from congenital heart surgery at the Bristol Royal Infirmary. Lobectomy for lung cancer was chosen as the marker operation for thoracic surgery because it is the only major thoracic operation performed often enough for meaningful statistical comparisons to be feasible. It is the only general thoracic operation for which data are specific to a surgeon, and data are now available covering two years.
Questions arise as to how such data should be presented and whether the reporting of such data provides a useful tool for assessing the quality of thoracic surgery.1 We aimed to calculate the mortality from this procedure and in relation to individual surgeons. No other details were available on which to base a risk adjustment. We knew nothing about the patients, such as age or sex, only whether they had left hospital alive or died.
Studies of other surgical procedures have shown a link between how frequently a surgeon performs a procedure and survival of the patient to leave hospital and at five years.2 3 The possibility of such a volume effect is of importance in UK thoracic surgery as around half of lobectomies are performed by cardiothoracic surgeons, who perform lobectomy infrequently compared with surgeons who concentrate on lung surgery (table 1). We explored the relation between in-hospital mortality of patients undergoing lobectomy for primary lung cancer and the number of lobectomies performed by the surgeon. From the pooled data it was not possible to identify any individual surgeon, and this was not our purpose. We wanted to highlight some of the pitfalls of interpreting in-hospital mortality in relation to individual surgeons.
Data were collected by a designated member of each thoracic surgical unit (surgeon or data manager) as part of the UK Society of Cardiothoracic Surgeons' annual return and collated centrally.4 These returns are not audited externally, but the accuracy of the computer entry is verified by each unit. Operations are categorised under 237 headings and pooled for the unit. Postoperative deaths during an admission for a procedure are recorded, but not the cause of death.
We present data on 4028 patients who had undergone lobectomy for primary lung cancer by one of 102 surgeons reporting in both of the financial years 1999-2000 and 2000-1. Firstly, we calculated the overall mortality from lobectomy with 95% confidence intervals, then we produced a scatter plot of mortality in relation to the number of procedures performed by the surgeon over the two year period. We also plotted absolute mortality over the two years in relation to the number of procedures performed by the surgeon during that time. As a guide, we added three lines to the plot: one corresponding to the overall average in-hospital mortality and two between which 95% of points would be expected to lie if the risk of in-hospital mortality were the same for all patients. We divided the data into fifths according to the number of operations performed by the surgeon over the two years. For each fifth, we calculated the mortality and exact 95% confidence intervals and used aχ2 test to determine any significant differences in in-hospital mortality between the five groups.2
Of the 4028 patients who had undergone a lobectomy over the two year period, 103 died (mortality 2.6%, exact 95% confidence interval 2.1% to 3.1%) during the same hospital admission. Figures 1 and 2 present the data for each of the 102 surgeons. Figure 1 gives an initial impression of a relation between surgical volume and risk. Most of the data in figure 22 lie within the 95% limits that would be expected if the risk of in-hospital mortality were the same for all patients, regardless of surgeon or other factors. Table 2 presents the mortality among the five groups of patients defined by the number of procedures performed by the surgeon over the two years.
To give a roughly equal number of patients in each group, we allocated the patients of two surgeons that had each performed 64 operations to different groups. Three deaths were related to one of the surgeons and none to the other. At the first analysis the split was made according to the sequence in which the surgeons' cases were listed. We found no significant differences in in-hospital mortality between the five groupsχ2=1.477, df=4, P=0.83).
In-hospital mortality after lobectomy for primary lung cancer is not related to the number of procedures performed by an individual surgeon. The relation between a surgeon's volume and outcome has attracted interest, and it seems reasonable to believe that there could be an effect. Data have been published on 2.5 million operations over five years in the Medicare system.2 These data, which were analysed with the model we used, show powerful effects for pancreatic and oesophageal cancer, which also apply to pneumonectomy and lobectomy for lung cancer. Five year survival is also influenced by the number of lung resections for cancer performed by a surgeon.3 This may be a learning curve or practice effect.5 It may also reflect better decisions made by teams dealing with lung cancer rather than those whose major focus is cardiac.
The UK data, however, do not show a relation between volume and outcome. Patients in the first group, who were operated on by 49 surgeons with an annual volume of one to 15 lobectomies, compared well with patients in the fifth group who were operated on by just six surgeons doing 47 to 96 lobectomies a year. Across the five groups the mortality varied from 2.0% to 2.9%, with no evidence of relation to volume.
Most of the 102 surgeons we reviewed performed both heart and lung surgery.6 7 Lobectomy for lung cancer was thus a small proportion of their work. They typically performed around 150 to 200 cardiac operations a year, so any practice effect for lobectomy (an operation of intermediate technical difficulty in cardiothoracic surgery) is likely to be subsumed under the overall case mix. It is also likely that any volume effect according to setting is lost because the lower volume of lobectomies are most commonly performed within cardiothoracic units, which deal with a large volume of major chest surgery.2 In the United Kingdom we need the contribution of “low volume” surgeons (surgeons performing less than 24 lobectomies a year provide 40% of the UK service for this procedure), so our results are reassuring.7
Figure 1 gives the impression of a relation between surgical volume and outcome. This can be explained by the eye being drawn to the single high mortality figure of 20%, representing one death out of five cases, and not to the 35 of 59 points along the x axis which represent surgeons performing fewer than 20 lobectomies a year. It is more than likely that at least one surgeon with a low surgical volume would display high mortality purely by chance–for example, if all the patients of all the surgeons faced the same risk of in-hospital death, there would be a 65% chance of at least one of the surgeons who performed five or fewer operations over the two years having a mortality of 20% or more. Similarly deceptive is the way in which the points line up in asymptotic curves. All points representing a single death describe a curve running from 20% to about 2%, as the denominator increases from five to over 50. The same effect is seen as the points for surgeons with two and three deaths line up. The major characteristics of figure 1 are therefore artefacts rather than useful information about the performance of individual surgeons. Figure 2 provides a graphical representation of the data that is less open to misinterpretation. Importantly, the low level of absolute mortality is more obvious than in figure 1.
What is already known on this topic
British cardiothoracic surgeons must report their mortality figures for “marker” operations
Lobectomy for primary lung cancer is the marker operation for thoracic surgery; half are performed by surgeons who perform the procedure infrequently
Studies of other surgical procedures have shown a link between surgical volume and survival to leave hospital
What this study adds
In-hospital mortality after lobectomy is not related to the number of procedures performed by a surgeon
Mortality figures derived from a small number of cases are unreliable and should not be used as the basis of important decisions made by patients
Mortality from lobectomy for primary lung cancer is a poor means of measuring surgical performance
So what purpose does the collection of surgeon specific data serve? Annual surgical volumes are too small to form the basis of any statistically valid comparison of one surgeon with another. It is also likely that anaesthesia and postoperative care are as important as surgical competence in determining survival after lobectomy. The more important determinants of survival are patient related and include age, pulmonary function, cardiac status, and other comorbidity. And yet the data collected on lobectomy are not adjusted for risk. The development of risk scoring within thoracic surgery is clearly needed.
Previous pooling of data by the Society of Cardiothoracic Surgeons was based on anonymity. The current system is open to dishonesty and the likelihood of surgeons refusing to operate on high risk cases. For these reasons we believe that the registration of lobectomy data is an inadequate means of measuring surgical competence. Lung cancer is a rapidly fatal disease, and an informed patient might well choose to face a substantial surgical risk for the chance of cure.8 A better indicator of quality would be cancer free survival at five years,3 and this would reflect better the performance of the cancer multidisciplinary team, who are jointly responsible for patient selection.
The members of the UK Society of Cardiothoracic Surgeons voluntarily contribute their data. The content of this paper was presented to the UK Thoracic Forum on 9 February 2002, and the members present unanimously agreed to these data being presented. The interpretation and opinions are those of the authors.
Contributors TT is the surgeon responsible to the Society of Cardiothoracic Surgeons of Great Britain and Ireland for the data on thoracic surgery. He prepared the first draft and worked on all editions of the manuscript. He will act as guarantor for the paper. MU prepared the data displays, performed all statistical analyses, and worked on all versions of the manuscript. AB collates the data on thoracic surgery and worked on all versions of the manuscript.
Funding AB is paid by the Society of Cardiothoracic Surgeons of Great Britain and Ireland to collate the data, but his academic contribution to this work was not funded.
Competing interests None declared.