Improving mortality of coronary surgery over first four years of independent practice: retrospective examination of prospectively collected data from 15 surgeonsBMJ 2004; 329 doi: https://doi.org/10.1136/bmj.38173.577697.55 (Published 19 August 2004) Cite this as: BMJ 2004;329:421
- Ben Bridgewater, consultant cardiac surgeon ()1,
- Antony D Grayson, regional clinical information analyst2,
- John Au, consultant cardiothoracic surgeon4,
- Ragheb Hassan, consultant cardiac surgeon3,
- Walid C Dihmis, consultant cardiac surgeon2,
- Chris Munsch, consultant cardiac surgeon5,
- 1 South Manchester University Hospital, Manchester M23 9LT
- 2 Cardiothoracic Centre, Liverpool L14 3PE
- 3 Manchester Royal Infirmary, Manchester M13 9WL
- 4 Blackpool Victoria Hospital, Blackpool FY3 8NR
- 5 Leeds General Infirmary, Leeds LS1 3EX
- Correspondence to: B Bridgewater
- Accepted 14 June 2004
Objective To study the “learning curve” associated with independent practice in coronary artery surgery.
Design Retrospective analysis of prospectively collected data.
Setting All NHS centres in north west England that carry out cardiac surgery in adults.
Participants 18 913 patients undergoing coronary artery surgery for the first time between April 1997 and March 2003, 5678 of whom were operated on by 15 surgeons in the first four years after their consultant appointment.
Main outcome measures Observed and predicted mortality (EuroSCORE) for surgeons in their first, second, third, and fourth years after appointment as a consultant compared with figures for established surgeons.
Results Overall mortality decreased over the six years of study (P = 0.01). Of the patients operated on by established surgeons or newly appointed consultants, 265/13 235 (2.0%) and 109/5678 (1.9%), respectively, died (P = 0.71). There was a progressive decrease in observed mortality with time after appointment as a consultant from 2.2% in the first year to 1.2% in the fourth year (P = 0.049). This result remained significant after adjustment for time and case mix (P = 0.019).
Conclusions Mortality in patients operated on by newly appointed consultant surgeons is similar to mortality in patients operated on by established surgeons. There are significant decreases in crude and risk adjusted mortality in the four years after appointment. These findings should influence the nature of practice in newly appointed surgeons.
In British medicine consultants are appointed to hospitals after a defined period of training, once they have satisfied national training bodies and been successful in open competition for an advertised post. NHS consultants are independent practitioners who function under clinical governance systems in the institutions in which they are employed. Increasingly, they are also regulated by national initiatives, such as the planned publication of surgeon specific mortality data for coronary artery surgery in the United Kingdom in 2004.1
In a recent study we looked at the outcomes of coronary artery surgery for individual surgeons in north west England.2 An incidental finding was a strong association between the volume of operations each surgeon had performed and mortality. This may be due to a “learning curve” effect, with higher mortality in patients operated on by newly appointed surgeons, starting late in the study period. We studied this in more detail by looking at changes in case mix and mortality after new consultant appointments in cardiac surgery in north west England.
The north west quality improvement programme in cardiac interventions is a regional consortium involving all four NHS centres (Blackpool Victoria Hospital; Cardiothoracic Centre, Liverpool; Manchester Royal Infirmary; and South Manchester University Hospital) that perform cardiac surgery and percutaneous coronary interventions in adults in north west England. The group aims to improve continuously the quality of care for patients receiving cardiac interventions using a regionally based systems approach.3
We prospectively collected data on 18 913 consecutive adult patients undergoing cardiac surgery between 1 April 1997 and 31 March 2003 in north west England. We included only those patients undergoing isolated coronary artery surgery (excluding repeat surgery). For each patient we collected data on preoperative and operative variables to calculate predicted mortality. Data were collected in each institution and returned to a central source for analysis. Validation of data was conducted in each centre. Mortality was defined as any death in hospital. Every patient record contained an identifier for each consultant surgeon.
Design of the project
We addressed four specific questions:
Are there differences in observed and predicted mortality between patients operated on by newly appointed surgeons and their more experienced colleagues?
Are there changes in observed and predicted mortality in patients operated on by newly appointed surgeons over the first four years of appointment?
If there are improvements in risk adjusted mortality over the first few years of appointment, what is the time course of those changes?
If there are decreases in mortality, are they seen in low or high risk patients?
The north west regional audit project started in 1997. Since that time there have been 13 new appointments. In addition, two surgeons practising in 1997 were in the first four years of their consultant appointment. For the analysis we pooled surgeons into four groups according to year of independent practice (first, second, third, and fourth years). We included all temporary appointments of less than a year as first year appointments. Associate specialists who had been in independent practice for more than four years by the time of their consultant appointment were not included as new appointments; if they had been in independent practice for less than four years they were included in the appropriate year as a new consultant. We also analysed data excluding locum appointments from the dataset.
Categorical data are shown as percentages with 95% confidence intervals, while continuous data are shown as means. We used χ2 tests for trend and Wilcoxon rank sum tests as appropriate to test for significant differences between groups. Crude mortality was determined for each patient and a predicted mortality calculated by using additive EuroSCORE.4 If a factor necessary to calculate the EuroSCORE was missing in the record, that factor was assumed to be absent (this occurred in less than 2% of cases). While the mean EuroSCORE is often given as an indication of operative risk of a group, the score is not normally distributed and so we have given results as means and medians with 25th and 75th centiles. To investigate in which type of patients the observed changes occurred, we subdivided patients into low (EuroSCORE < 6) and high (EuroSCORE < 6) risk groups for further analysis.2 To adjust for a potential time effect we constructed a logistic regression model on all cases (n = 18 913) that included the financial year of the operation (treated as 1997-8 = 1, 1998-9 = 2, etc) and the EuroSCORE.5 Using the logistic model, we then calculated risk adjusted mortality percentages for the first four years of independent practice, in line with the system developed in the state of New York.6 The results reflect performance adjusted for time and case mix.
A total of 18 913 patients underwent surgery in the region during the study period, 5678 of whom were included in the learning curve analysis. Table 1 shows details of the new consultant appointments and number of patients operated on. The number of patients pooled into the first, second, third, and fourth years were 1967, 1534, 1283, and 894 respectively. The average number of patients per surgeon was 379. The learning curve dataset included 30% of the total number of patients undergoing surgery.
Overall, 374 patients (2.0%) undergoing isolated coronary artery surgery during the study period died. Mortality in patients operated on by surgeons in their first four years after appointment and that of established consultants was similar (109/5678 (1.9%, 95% confidence interval 1.6% to 2.3%) and 265/13 235 (2.0%, 1.8% to 2.2%), respectively, P = 0.71). The mean predicted mortality for patients operated on by consultants in the first four years after appointment (EuroSCORE mean 3.1, median 3, 25th and 75th centiles 1 and 4) was similar to that of those operated on by more established surgeons (mean 3, median 3, 25th and 75th centiles 1 and 4) (P = 0.79).
The ratio of observed to expected mortality is the best risk adjusted summary of performance. The ratios for the patients operated on by surgeons in the first four years of practice and more established surgeons were 0.61 and 0.66, respectively.
There was a progressive decrease in observed mortality during the first four years after appointment, from 2.2% (1.6% to 3.0%) to 1.2% (0.5% to 2.1%) in the first and fourth years, respectively (P = 0.05, table 2). The predicted mortality progressively increased over the four years from a mean EuroSCORE of 3.0 to 3.3 (P = 0.03). The ratio reduced by a factor of 50% from the first to the fourth year after appointment (P < 0.001). Exclusion of locum appointments from the dataset had no effect on the results. After we accounted for the effect of time and case mix by using the logistic model, the adjusted mortality decreased from 2.3% in the first year to 1.0% in the fourth year (P = 0.019, table 2).
Table 3 shows changes in mortality in the low and high risk groups. There was a significant reduction in mortality in the low risk group with increasing length of time since appointment that was not seen in high risk patients. Table 4 shows that for established consultants there was a significant reduction in mortality over the time of the study (P = 0.01).
We have shown significant improvements in crude and risk adjusted mortality associated with coronary artery surgery over the four years after surgeons are appointed to independent practice. The observed mortality for surgeons in their first and second years after appointment was slightly higher than for established consultants but lower in the third and fourth years. These improvements were due to better outcomes in patients undergoing low risk surgery.
Strengths and weaknesses of the study
Our study has been conducted on a dataset produced from all patients undergoing cardiac surgery in north west England since 1997. The data have been subjected to local validation and have the confidence of clinicians but have not been subjected to external validation, which is a weakness. We consider that the number of surgeons, the large population of patients, and the fact that these appointments were across four different hospitals add strength to our study. We were initially concerned that some of the learning curve effect detected may have been due to overall changes in mortality because of progressive improvements in outcome with time, which is an effect that has been clearly seen in previous work.7 The improvements we have shown over the first four years of practice were still significant after we adjusted for time and case mix by logistic regression. We have looked only at mortality and not at morbidity, which may be a more sensitive indicator of subtle improvements in the quality of care.
We observed a significant decrease in crude mortality as surgeons become more experienced. We used a well accepted model of risk prediction, the additive EuroSCORE,4 to look for changes in case mix to see if alterations in predicted mortality might explain the observed mortality. The EuroSCORE is generally accepted to be a good but not perfect predictor of mortality.2 8 9 Predicted mortality increased in the four years after appointment, suggesting that surgeons are operating on patients with increasing comorbidity, which confirms local clinical perceptions. Use of the ratio of observed to expected mortality as an overall indicator of risk adjusted quality of care highlights this effect of decreasing mortality despite increasing complexity of case mix. While there must be some caution exercised when using risk prediction models, it would seem to be appropriate in this setting, but it is conceivable that more experienced surgeons may become “smarter” at exploiting limitations within the risk prediction model, which would not affect our observations on crude mortality but may potentially confound the risk adjusted analysis.
The mortality in patients of surgeons in the first two years of appointment was slightly higher than that in patients of more established consultants performing surgery during the time period studied. The observation that mortality in patients of surgeons in their third and fourth years of appointment is below that seen in patients of more established surgeons (both crude mortality and risk adjusted outcomes), suggests that either the performance of these consultants is better than their peers, or that there are complex issues associated with case mix and higher risk surgery that may account for these differences but were not detected with our methods.
Comparison with other studies
There are few studies on learning curves in surgery. There have been reports of learning curve effects with new techniques or with new complex operations in both cardiac10 11 and non-cardiac surgery.12–17 There are thought to be three separate components of the learning curve for new procedures: the specialist community, the institution, and the individual surgeon.11 We examined the effects of the learning curve in surgeons on a well established procedure for which they have been trained. Most trainees will have performed many hundreds of coronary artery bypass operations before appointment in hospitals with existing skills and infrastructure, so we are not looking at aspects of learning curves of the specialist community or institution. We saw significant improvements in risk adjusted mortality in the four years after appointment, which is an “individual surgeon” effect. We are not aware of previous reports on learning curve effects associated with moving from training grade to independent practice.
A recent study on learning curve effects in complex laparoscopic colorectal surgery14 suggests a steady state of technical competence is achieved after 70-80 procedures. The number of operations needed to achieve the “plateau phase” for technical issues in coronary artery surgery is probably similar. As all new appointees in this study would have performed well over 100 operations, while there may still be ongoing technical improvements after appointment, the improvements we have detected are probably related to issues other than primary technical ability.
If the improvements seen after appointment are not related to technical ability it implies that surgeons improve the non-technical aspects of surgery, including decisions on case selection and aspects of preoperative, intraoperative, and postoperative care and decision making. Our study suggests that patients and hospitals can be reassured that mortality is not higher in patients of newly appointed surgeons, but initiatives that may modify the learning curve will have advantages for patients, hospitals, and surgeons.
There are several possible ways to minimise the effects of learning curves: improve training, increase supervision of new appointments, and introduce other strategies to accelerate improvement. The effect seen here was detected in surgeons who have been trained under existing structures (the tiered registrar/senior registrar system or seamless Calman training) and will have spent in excess of five years as specialist trainees. Plans to shorten the surgical training period, along with a reduction in training hours due to implementation of the European working time directive, may have implications on the experience of new consultants in the future, which would act to increase possible learning curve effects unless other modifying influences are introduced.
It has been suggested that learning curve effects in medicine are similar to those in manufacturing17 and that by applying learning curve theory, higher quality health care may be achieved. Learning curve theories accepted in manufacturing are that outcomes improve with higher volume due to learning, increases in improvement become smaller with progressive experience, and increases in improvement follow a predictable pattern. In previous studies of learning curves in cardiac10 11 and non-cardiac12–17 surgery, effects have all been detected on a relatively small number of cases. We have observed a learning curve effect in surgeons who are performing high volumes of surgery. While we saw continuous improvements in each of the first four years of appointment, the most marked improvements were between the third and fourth years, which is at odds with data from manufacturing. The annual volumes of coronary artery surgery in units and individuals are high, which differs from the typical small number of cases undertaken in other surgical specialties.
Volume of experience, incentives, and managerial factors are potential modifying influences on the learning curve.18 It seems inappropriate to increase the volume of surgery to higher than current levels. Planned publication of surgeon specific mortality and use of performance data in clinical excellence awards may well influence learning curves in future but may also encourage newly appointed surgeons to turn down higher risk cases, unless robust risk adjusted measures are used for performance management. Managerial input—including the use of clinical pathways, robust clinical risk management strategies, effective supervision, and targeted continuing professional development—would seem to be the most effective way to accelerate learning curve effects. Experienced clinical support for new appointments, not just for technical issues but for all aspects of clinical practice, including case selection and perioperative management, seems sensible.
Unanswered questions and future research
Our study raises several questions that need to be addressed in future studies. We do not know whether causes of death change as surgeons become more experienced, which may help to guide plans to improve outcomes in newly appointed surgeons. While the effect for cardiac surgery seems to be marked, we do not know whether the learning curve is restricted to this specialty or whether it is seen in other surgical specialties or is a widespread phenomenon throughout medicine. There are several initiatives that are leading to a reduction in hours during the working week and limiting the number of years of training for surgeons, and we do not know if this will make the effects we have seen more marked. Finally, we do not know whether initiatives such as a structured supervision and targeted continuous professional development would accelerate the learning curve effect.
What is already known on this topic
Learning curves are associated with the introduction of new procedures in medicine
Although mortality for coronary artery surgery has decreased over time, it is not known whether mortality among patients of new consultants changes as the doctors become more experienced and whether such changes are independent of overall improvements of their peers
What this study adds
In the first four years after the appointment of UK consultant surgeons to independent practitioner status outcomes significantly improve
While initial mortality associated with newly appointed consultant surgeons is not significantly higher than that for more established consultants, there is a 50% reduction in risk adjusted mortality over the subsequent four years
This study has been conducted on behalf of the north west quality improvement programme in cardiac interventions, and the consultant surgeons involved were John Au, Ben Bridgewater, Colin Campbell, John Carey, John Chalmers, Walid Dhimis, Abdul Deiraniya, Andrew Duncan, Brian Fabri, Elaine Griffiths, Geir Grotte, Ragheb Hasan, Tim Hooper, Mark Jones, Daniel Keenan, Neeraj Mediratta, Russell Millner, Nick Odom, Brian Prendergast, Mark Pullan, Abbas Rashid, Franco Sogliani, Paul Waterworth, and Nizar Yonan. Narinda Bhatnagar, Albert Fagan, Bob Lawson, and Udin Nkere. Peter O'Keefe, Richard Page, Ian Weir, and David Sharpe left the collaboration during the study period. We thank the audit officers working in each centre for their hard work in collecting and validating the data and Mark Jackson for his analytical advice.
Contributors BB had the idea for the study and along with ADG was responsible for the study design and analysis. All authors contributed to preparation of the manuscript. BB is guarantor for the study.
Funding Funding for the programme was received from all primary care trusts in north west England. All authors were independent from the funding.
Competing interests None declared.
Ethical approval Ethical approval for the north west quality improvement programme was obtained from the regional ethical committee.