Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Should we be reassured?
- Asif Hasan (14 March 2005)
-
Risk-stratified outcome following CABG surgery
- Simon P McGuirk (14 March 2005)
-
What are outcomes?
- Alastair J Macdonald (15 March 2005)
-
Should anaesthesia, intensive care, cardiology and gastroenterology be divisions of surgery?
- RICHARD G Fiddian-Green (17 March 2005)
-
Comparing cardiac mortality rates
- Roy J. Shephard (1 April 2005)
-
Calculating control limits in funnel plots
- David J Spiegelhalter (7 April 2005)
|
|
|||
|
Asif Hasan, consultant paediatric cardiac surgeon Freeman hospital, Freeman road. NE7 7DN
Send response to journal:
|
Editor- Ben Bridgewateret al are to be congragulated for again highlighting the issue of satisfactory performance by the cardiac surgeons. However, we in the profession already knew that. Presumably the Guardian newspaper has also realised that there are no "bad apples" to give "legs" to the story,as we have not seen in print the article which was reported to be published last week as suggested by an accompanying editorial by Tom Treasure. Should we be then reassured that there is no prospect of a system failure to repeat the Bristol saga. The answer I am afraid cannot be -no. The Kennedy Report was about system failure in a paediatric cardiac unit. Five years hence there is little to show for the millions spent on the report. The key recommendation regarding the size of a paediatric unit ie undertaking 300 heart operation was rubbished by the minister and not accepted. The recommendations which were accepted are far from implemented.The reports of similar survival amongst all paediatric cardiac centers only hides the underlying problems some units are facing with lack of facilities, intensive care beds and lack of medical and nursing backup. The problems have been highlighted recently, publicly by Great Ormond Street Hospital, the flagship of paediatric cardiac services in UK. I feel that the fundamental recommendations of the Kennedy Report have been highjacked and resources are been directed into areas which we instinctively knew were never a source of concern regarding poor performance ie coronary artery surgery. I am not reassured by the reassurances I never needed. Competing interests: None declared |
|||
|
|
|||
|
Simon P McGuirk, Research Fellow Department of Cardiac Surgery, Birmingham Children's Hospital NHS Trust
Send response to journal:
|
Editor, I read with interest the recent article by Mr Bridgewater, which reported the in-patient mortality data for twenty-five named surgeons performing isolated first time coronary artery bypass graft (CABG) or aortic valve surgery in the North West [1]. While I broadly welcome this article, I believe that the data demonstrates a number of important additional features that were not addressed in the original article. Mr Bridgewater reported that, “all [of the surgeons] are performing to satisfactory standards”. Analysis of Fig 1 does show that the results of all 25 surgeons fall below the upper 99% confidence interval (CI) derived from the national data. However, the data also shows that the results of four surgeons [Duncan, Hasan, Hooper and Sogliani] are better than expected, falling below the lower 99% CI. The interpretation of crude mortality data must be undertaken with caution, because it fails to account for variation in case mix. For this reason, Mr Bridgewater has published the mortality data following cardiac surgery in two distinct strata, low and high-risk, based on the additive EuroSCORE [2]. While the additive EuroSCORE remains a simple ‘gold standard’ for risk assessment in adult cardiac surgery in Europe, it may have been more appropriate to have used the logistic EuroSCORE [3], which continues to predict mortality very accurately even amongst patients in the very high-risk category. The logistic EuroSCORE would have enabled analysis of risk-adjusted outcome, rather than simply risk-stratified outcome. One of the primary objectives of this article appears to have been to undertake a comprehensive and well-informed analysis of risk-stratified data. However, the only comparative analyses were made using crude mortality (Fig 1 and 2). There is no current risk-stratified national data available for comparison because approximately a third of patients have incomplete data precludes calculation of the EuroSCORE [4]. Nevertheless, it is possible to make some more detailed analyses, based on the data presented. It is possible, for example, to compare the proportion of high- risk cases, as well as the outcome following both low-risk and high-risk surgery. For CABG surgery, approximately 17% (range, 7 – 24%) of cases were considered high-risk. A funnel plot with 95% and 99% CI of this proportion (Fig A), calculated using the method of EB Wilson [5], illustrates that the case mix was generally comparable for all surgeons. One surgeon [Hooper] had a lower proportion of high-risk cases, which may explain, in part, why this surgeon appeared to perform better than the national average.
It is also possible to construct similar funnel plots for the outcome following low-risk and high-risk CABG surgery (Fig B and C, respectively). These plots illustrate that all the surgeons performed at least as well as the entire cohort, in both risk strata. These plots also identified two surgeons [Duncan and McLaughlin] who performed better than expected for low-risk surgery and four surgeons [Duncan, Hooper, McLaughlin and Sogliani] who performed better than expected for high-risk surgery.
Whether you agree with the principle of reporting surgeon-specific data or not, the article by Mr Bridgewater has placed this data firmly within the public domain. It is imperative that this data is analysed rigorously. Only in this way will good practice, like that evident in this article, be identified and promoted. [1] B Bridgewater. Mortality data in adult cardiac surgery for named surgeons: retrospective examination of prospectively collected data on coronary artery surgery and aortic valve replacement. BMJ 2005; 330:506- 10. [2] SAM Nashef, F Roques, E Gauducheau et al. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9-13 [3] P Michel, F Roques, SAM Nashef et al. Logistic or additive EuroSCORE for high-risk patients? Eur J Cardiothorac Surg 2003;23:684-687 [4] BE Keogh, R Kinsman. Fifth National adult cardiac surgical database report 2003. Henley-on-Thames: Dendrite Clinical Systems, 2004. [5] EB Wilson. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 1927;22:209-212. Competing interests: None declared |
|||
|
|
|||
|
Alastair J Macdonald, Chair, Clinical Outcomes Group, SL&M NHS Trust Ladywell House, Lewisham, London SE13 6JZ
Send response to journal:
|
The two main tables in this excellent paper are headed "Outcome". Outcomes are usually defined as changes in health status that may be attributable to an intervention in a defined group of patients. Although post-operative death is clearly an outcome, simply avoiding it is surely not the main aim of surgery. (If this were true, the surgeon who instead of operating waved his arms about might appear the best.) To understand outcomes we must also routinely gather data on how much patients improve after surgery or any other interventions- a procedure pioneered by Codman (1). Although this is much more challenging than just recording severe adverse outcomes, without it we are still in the dark. Reference 1. Kaska SC, Weinstein JN. Historical perspective. Ernest Amory Codman, 1869-1940. A pioneer of evidence-based medicine: the end result idea. Spine 1998;23(5):629-33. Competing interests: None declared |
|||
|
|
|||
|
RICHARD G Fiddian-Green, FRCS, FACS None
Send response to journal:
|
The most comprehensive analysis of cardiac surgery I have seen appeared in the Guardian yeterday. From my recollection of their figures the mortality, in the hands of those surgeons and their teams who had done enough cases for the figures to be meaningful, ranged from some 0.3£ to 12%. That is a huge difference [some 4000%]not evident in this analysis (1). Of particular interest in the Guardian data was the difference in outcome for "low-risk" [less than 1%] and high-rish" [about 12%] cases. For one surgeon, and possibly a few more, there ws no difference between low-risk and high-risk patients and the mortality for both was less than some 1%. This is what is seen in general surgery in the US, a huge gulf in abilities between some operating in some select centers and the rest. Low-risk cases are very poor discriminators of ability and since they form the bulk of the cases in analyses such as this (1) they can conceal real and big differences. One expects outliers in a bell-shaped curve but these individuals are in a completely different league. The conclusion that, "Mortality for all surgeons fell below 99% control limits of the national mean for both operations" conceals this. It also avoids the need to withdraw operating privileges from any consultant cardiac surgeon who has been trained in the NHS or closing any cardiac center within the NHS. It is easy to conclude that the average mortality of 1.8% for coronary surgery and 1.9% for aortic valve surgery are outstanding results until one considers the number of avoidable complications and deaths involved. From 1 April 2001 to 31 March 2004 10 163 patients had cardiac surgery by 25 consultant surgeons operating in four centers in the northwest of England, an average of 121 CABG and 14 aortic valves per surgeon/year. There were some 200 deaths of which some 150 might have been avoided had the had their operations been dome by a surgeon in the top league. What of avoidable morbidity, such as cognitive changes and strokes? It must have been much higher. A top league surgeon working in a US heart factory line must do some 2-3 CABGs three times a week 45 weeks a year. That is some 300 cases a year and 900 in three years. Of these a higher proportion might be "high- risk" cases, many of them re-dos, tha in the present study. Exposure to this kind of workload, certainly for a period, gives surgeons on their growth curve the opportunity to enter the top league. One solution to the need for more surgeons in the top league might be to halve the number of cardiac surgeons and give each 2 to 3 times more cases to do each year. That raises the prospect of having a pyramidal system in wich those of lesser surgical abilities have to be shifted into other positions, notably cardiac anaesthesia and intensive care. That possibility does not exist today without retraining. My guess is that many would do a superior job in anaesthesia and intensive care by virtue of their surgical traning. Having allowed anaesthetists and even medical intensivists to assume responsibility for perioperative care seems to have been a retrogressive development (2). I venture to suggest that surgery, anesthesia and intensive care might all be better served if made divisions of surgery. The same may be said for all medical specialties offering invasive treatments today includng cardiology, gastroenterology and even radiology and dermatology. All might be far better served by surgeons who were were eithr unable to enter the top league or had no desire to do so. A solution as radical as this might be needed to resolve the destructive competition that exists between those who follow the staff of Asclepius and those who follow the staff of caduceus today(3). One adantage of this proposal is that it would provide the attractive opportunity for the many frustrated surgeons who have found themselves locked into non-surgical disciplines of doing surgery and even entering the top league should they find they have the motivation and aptitude. It could be a win-win situation. It should also certainly eliminate the problems that have been caused by the widespread violation of established sirgical principles by those delivering invasive treatments in non- surgical disciplines. It should also make surgeons far more attentive to what is happening above the ether screen. It many cases it has become a Berlin wall that has divided objctives, practices and even loyalties to the detrement of patients. 1. Ben Bridgewater on behalf of the adult cardiac surgeons of north west England Mortality data in adult cardiac surgery for named surgeons: retrospective examination of prospectively collected data on coronary artery surgery and aortic valve replacement BMJ 2005; 330: 506-510 2. DR. RICHARD FIDDIAN-GREEN, FRCS, FACS November 21, 2002 MEDICAL MONUMENTS TO FAILURE www.redflagsweekly.com/fiddian_green.html 3. Caduceus vs Staff of Asclepius. drblayney.com/Asclepius.html Competing interests: None declared |
|||
|
|
|||
|
Roy J. Shephard, Professor Emeritus of Applied Physiology, University of Toronto PO Box 521, Brackendale, BC V0N 1H0, Canada
Send response to journal:
|
In recent years, the British Medical Journal has had an enviable record in bringing statistical rigour to the analysis of surgical and medical issues. I was thus somewhat surprised to note that in the recent article by Ben Bridgewater, the survival rates achieved by various cardiac surgeons in North-western England are compared without reference to the 95% confidence limits of the data. This issue is particularly critical in Table 2,where the data for individual surgeons is based largely upon experience gleaned from only zero to five patients. Those surgeons who have not as yet attempted a particular operation achieve the enviable record of 100% survival rates! However, if the two tables in this paper were to be presented after calculation of 95% confidence limits, I think that the paper would demonstrate few statistically meaningful differences in the performance of individual surgeons. Competing interests: None declared |
|||
|
|
|||
|
David J Spiegelhalter, senior scientist MRC Biostatistics Unit Cambridge CB2 2SR
Send response to journal:
|
Bridgewater et al [1] use funnel plots to compare individual surgeons’ mortality rates with a target, but unfortunately the limits have been calculated incorrectly. Confidence intervals around the ‘target’ have been plotted, whereas the correct control limits should be prediction intervals under a Binomial distribution with the target mean, with which the observed proportion can then be directly compared [2]. These are more stringent than the limits plotted by Bridgewater et al, but still just include all the surgeons for the overall AVR mortality (whether this remains the case for the high-risk AVR cases depends on the assumed target). Any observation lying outside the 99% intervals should reject the hypothesis that the true underlying mortality rate is ‘on-target’ with a one-side P-value of 0.005: the surgeon with 5 deaths out of 52 cases (10%) has a P-value of 0.012 relative to a target of 3% mortality (using the convention that the one-sided P-value is the chance under the target rate of a more extreme outcome plus ½ the chance of the observed outcome), and hence is just inside the 99% control limits. Funnel plots are a useful technique but some care is required in setting the control limits. References [1] Bridgewater B; Adult Cardiac Surgeons of North West England. Mortality data in adult cardiac surgery for named surgeons: retrospective examination of prospectively collected data on coronary artery surgery and aortic valve replacement. BMJ. 2005 ;330:506-10 [2] Spiegelhalter DJ. Funnel plots for comparing institutional performance. Stat Med. 2005;24:1185-1202 Competing interests: None declared |
|||
