Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Paediatric cardiac surgeons are not in this alone
- Francis E Arnstein (8 October 2004)
-
Publicity and performance change in Bristol
- Stephen N Bolsin (10 October 2004)
-
Paediatric Cardiac Surgical Mortality
- James Morris, Nick Archer, Consultant Paediatric Cardiologist (11 October 2004)
-
Damned for rumours and low quality data
- Andrew D McLean (12 October 2004)
-
Re: Paediatric cardiac surgeons are not in this alone
- Nesamani K.S.Vengadasalam, 08300 Malaysia (12 October 2004)
-
Clarification of 'adjusted rates' is required
- Gareth J Parry, Elizabeth S Draper, Patricia McKinney (19 October 2004)
-
Paediatric Cardiac Hospital Episode Statistics are unreliable
- John L Gibbs, David Cunningham, Marc de Leval, James Monro, Bruce Keogh (26 October 2004)
-
Author's reply
- Paul Aylin, Brian Jarman, Alex Bottle (1 November 2004)
-
Paediatric Cardiac Surgical Mortality
- James Morris, Trevor Campbell Davis (Chief Executive), Ravi Pillai, Stephen Westaby (Consultant Cardiac Surgeons), Satish Adwani, Nick Archer, Neil Wilson (Consultant Paediatric Cardiologists) (30 November 2004)
|
|
|||
|
Francis E Arnstein, Consultant Anaesthetist Western General Hospital Edinburgh EH4 2XU
Send response to journal:
|
EDITOR - The events at the Bristol Childrens Hospital have had far reaching consequences. The editorial team at the BMJ must consider how their headlines will be interpreted by other members of the media and the public. 'This week in the BMJ' ( 9th. October 2004 page 329) headlines 'Paediatric cardiac surgeons are getting better'. This implies that the results of their work is a consequence of their actions alone and is in contradiction to the conclusions of the article to which it refers. I believe it is most important that those not directly involved in the care of patients appreciate that in modern healthcare it is rare for the performance of one team member to be the only significant factor in patient outcome. Why not 'Paediatric cardiac surgical results are improving'? Competing interests: None declared |
|||
|
|
|||
|
Stephen N Bolsin, Director Division of Perioperative Medicine The Geelong Hospital, Ryrie St, Geelong, Vic, 3220
Send response to journal:
|
Dear Sir, I was interested to read the article in this week’s BMJ relating to the performance of paediatric cardiac surgery in the UK and specifically in the Bristol Royal Infirmary and the Royal Bristol Children’s Hospital [1]. The longitudinal comparison of the Bristol data with the rest of the country provides a demonstration of the change in performance that occurred in the Bristol unit after publicity surrounding the mortality data for that unit. This data provides confirmation of the previous published data that identified the excessive mortality for the Bristol unit prior to 1995 [2, 3]. However the data also confirms that concerns, expressed between 1990 and 1994, to the CEO and Chairman of the Trust Board (UBHT), Presidents and Vice Presidents of Royal Colleges, Dean of the Medical School, Professors of Surgery and Cardiac Surgery, Senior Medical Officers at the Department of Health, Departmental Directors and colleagues in the hospital had failed where publicity eventually succeeded [4]. The fact that the disaster could have been detected and hopefully prevented by the collection of routine data should make the collection and independent analysis of such data routine for all specialities in medicine [5, 6]. I am pleased that some good has come of the Bristol Cardiac Disaster, for children under 1 year of age requiring heart surgery in Bristol and their parents. I believe that I can take some credit for this improvement but I would also like to express my sympathy to those parents whose children died unnecessarily in Bristol before the mortality rate improved and apologise on behalf of the profession for our inadequate response to their children’s needs. Steve Bolsin Associate Professor Stephen Bolsin BSc MBBS MRACMA FRCA FANZCA DLitt (Hon) Director Division of Perioperative Medicine, Anaesthesia & Pain Management The Geelong Hospital Ryrie Street Geelong Victoria 3220 Australia References 1. Aylin, P., et al., Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991-2002. BMJ, 2004. 329(7470): p. 825-0. 2. Aylin, P., et al., Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984-96: was Bristol an outlier? Lancet, 2001. 358: p. 181-187. 3. Spiegelhalter, D.J., Mortality and volume of cases in paediatric cardiac surgery: retrospective study based on routinely collected data. BMJ, 2002. 324(7332): p. 261-262. 4. Bolsin, S.N., More on the Wisheart Affair. BMJ, 1999. 318: p. 1010 -11. 5. Spiegelhalter, D., et al., Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care, 2003. 15(1): p. 7-13. 6. Bolsin, S.N. and M. Colson, Making the case for personal professional monitoring in health care. International Journal for Quality in Health Care, 2003. 15: p. 1-2. Competing interests: A/Prof Bolsin worked at the Bristol Royal Infirmary for part of the time of the data collection |
|||
|
|
|||
|
James Morris, Medical Director ORH NHS Trust, John Radcliffe Hospital, Headley Way, Oxford OX3 9DU, Nick Archer, Consultant Paediatric Cardiologist
Send response to journal:
|
Editor The paper by Aylin and colleagues (1) on Paediatric Cardiac Surgical Mortality relies Hospital Episode Statistics (HES) data and prompts us to make some observations. The defects of HES data for clinical analysis have been widely acknowledged as considerable including by these authors themselves (2). These limitations are well illustrated by the striking contrast between the national infant open cardiac surgical mortality of 4% for epoch 6 (1999-2002) in Aylin and colleagues’ report and the 7.9% 30 day mortality for 2000/1 (3) derived from the Central Cardiac Audit Database (CCAD). HES data are, unlike CCAD, unvalidated and this discrepancy in national average mortality rates reflects significant underascertainment by HES. In more specific terms the absence of data from other UK centres (Edinburgh, Glasgow and Belfast) weakens the representative character of the data and we suggest it is inappropriate to group Harefield and Brompton Hospital results together as paediatric cardiac surgery was being performed in Harefield Hospital into epoch 6. This Centre has always submitted paediatric cardiac surgical information to the Society of Cardiothoracic Surgeons of the UK and Eire (SCTS) and, since its national inception, to CCAD. In addition, detailed audit information has been kept for internal use in Oxford. Not withstanding the authors’ opinion, we do consider that there are factors that could alter case mix in Oxford towards a higher risk. For example, in 1996-2000, there were a number of cases for which mortality in all centres is high (for example complex transposition) and with only a small total number of deaths such cases would alter overall survival rates. Also in a cardiac centre such as Oxford with on site maternity, neonatal and general paediatric services there were cases operated on that would not have survived to reach a more distant cardiac unit. On the eve of the publication of the Bristol enquiry in 2001, we were approached (as we believe were at least two other centres) by the Department of Health (DoH) and the Cardiac Tsar because of reports to them on the basis of HES data of an inappropriately high mortality for infant open heart surgery. Review of in house data and of information from SCTS on infant cardiac surgery outcomes was undertaken with the DoH and the Cardiac Tsar. It was concluded that if there was a downturn in performance in Oxford it related to transposition alone and that the Trust had recognized the possibility more than twelve months previously and acted appropriately. As Aylin and colleagues specifically mention the Oxford Cardiac Review (4) we consider it necessary to point out that the review, having looked at SCTS data from 1993 onwards with the help of independent paediatric and adult cardiac surgeons, made no adverse comments about clinical outcomes in either paediatric or adult cardiac surgery. Subsequently less detailed review of outcomes formed part of the Trust’s visit by NICE and by the Monro committee, neither of which identified a problem with paediatric cardiac surgical outcomes here. Finally, we should like to record that from 2000, our 30 day open infant cardiac surgical mortality has fallen progressively and for the year ending 31st March 2004 it was 4.2%. N Archer, Consultant Paediatric Cardiologist
References 1. Aylin P, Bottle A, Jarman B, Elliott P Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991-2002 BMJ 2004;329:825-7 2. Aylin P, Alves B, Best N et al Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984-1996: was Bristol an outlier? Lancet 2001;358:181-7 3. Gibbs JL, Monro JL, Cunningham D, Rickards A Survival after surgery or therapeutic catheterization for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database. BMJ 2004;328:611-5 4. Bridge S, Dussek J, Macpherson W Report of the external review into Oxford cardiac services. Issued by the South East regional Office. Leeds: NHS executive 2000 Competing interests: None declared |
|||
|
|
|||
|
Andrew D McLean, SpR, Cardiac Surgery Guy's Hospital, London, SE1
Send response to journal:
|
Dear sir, I write the same response for both articles regarding cardiac surgery outcomes in the UK. The data on cardiac surgical outcomes in this country are amongst the most detailed in the world. The Society of Cardiac and Thoracic Surgeons for the UK and Ireland publish annual reports (the blue book) every year based on quality assessed data. Now it seems that our speciality is to be judged on the basis of poor quality, incomplete data for paediatric cardiac surgical outcomes and rumours, yes rumours, for adult cardiac surgery. The rumour put out in one of the articles, whereby a cardiac surgeon carries out an unnecessary procedure on a patient for whatever ulterior motive, constitutes an accusation of gross professional misconduct by anyone's reckoning. Such an incident places a reponsibility to report this suspicion to the GMC. It is outrageous to suggest it in these pages. Why not forego all this complicated audit business and just chase us down the street with pitchforks and scythes? You can hang, draw and quarter me before I will have my clinical practice influenced by such drivel as this. Yours,
Competing interests: None declared |
|||
|
|
|||
|
Nesamani K.S.Vengadasalam, Hospital Director (medical) Hospital Sik, Sik, Kedah, 08300 Malaysia
Send response to journal:
|
I have been following this inquiry all along. Your heading seems most appropriate. Competing interests: None declared |
|||
|
|
|||
|
Gareth J Parry, Reader in Health Services Research University of Sheffield, Elizabeth S Draper, Patricia McKinney
Send response to journal:
|
The recent paper by Aylin and colleagues provides some interesting and potentially very perturbing results. The identification of a reduction in paediatric cardiac surgical mortality is very encouraging, however the identification of hospitals with excess mortality is a major cause for concern. Since 2003, all paediatric intensive care units in England and Wales have been contributing data on all admissions to the Department of Health funded Paediatric Intensive Care Audit Network (PICANet).[1] It may come as some re-assurance to families of patients treated in paediatric intensive care units in the UK that in the data thus far reported we have found no unit to have an unexpectedly high mortality rate. The method of analysis used by PICANet may be superficially similar to that used by Aylin and colleagues, but on closer inspection may be very different. PICANet produces mortality ratios that are carefully adjusted for the illness severity of children on admission to the unit using published risk adjustment tools.[2] This has the result that markedly different distributions of mortality ratios by unit are produced when plotting crude (unadjusted) mortality compared to unadjusted mortality. Thus, if one wishes to rank units according to their mortality, then units will rank in a very different order using crude compared to adjusted mortality. This is a phenomenon that has been seen to occur in many other areas when good quality risk-adjustment is applied.[3,4] It is curious therefore that in the paper by Aylin, the distribution of crude mortality seen in the supplemental figures is almost equivalent to that of the adjusted odds ratios seen in the main paper. Although, with perfect risk adjustment this can happen, a more likely and troubling cause could be the lack of valid and appropriate risk adjustment. There is no description of the methodology of the risk adjustment used in the paper by Aylin. We believe that it is vital for the NHS to identify areas of the service that are falling behind in performance. However, we also believe in the application of valid, reliable and robust scientific techniques for doing this. In order to provide re-assurance and therefore support to their work Aylin and colleagues must provide clear details of the adequacy of the risk adjustment tools they used in this study. 1. PICANet Annual Report May 2004, Universities of Leeds, Leicester and Sheffield 2. Shann F, Pearson G, Slater A, Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med 1997;23:201-7 3. Parry GJ, Gould CR, McCabe CJ, Tarnow-Mordi WO. Annual league tables of hospital mortality in neonatal intensive care: A longitudinal study. BMJ 1998; 316:1931-1935 4. Marshall EC, Spiegelhalter DJ, Sanderson C, McKee M Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates BMJ, 1998; 316: 1701 - 1705. Gareth Parry University of Sheffield Elizabeth S Draper University of Leicester Patrticia McKinney University of Leeds Competing interests: None declared |
|||
|
|
|||
|
John L Gibbs, Central Cardiac Audit Database lead clinician for congenital heart disease Leeds General Infirmary, LS1 3EX, David Cunningham, Marc de Leval, James Monro, Bruce Keogh
Send response to journal:
|
Editor We remain concerned by the lack of accuracy of the Hospital Episode Statistics (HES) used by Aylin et al (1) to analyse the paediatric cardiac surgical mortality in England after Bristol. Aylin’s group themselves reported that HES data “manifestly contains errors” and that their available data sources “have such clear limitations one could ask whether any reliable conclusions can be drawn” (2), and we have also shown HES to be inaccurate (3) for this complex specialty. The latest study from the Imperial College group (1) raises even more concerns over HES data, clearly illustrating that errors in HES are not consistent across the country. A centre with a high proportion of HES outcome returns (Oxford, for example) would almost inevitably identify more deaths than one with low returns, potentially giving a false impression of relative surgical performance. The Central Cardiac Audit Database (CCAD) collects, validates and analyses data from all UK paediatric cardiac units, centrally tracking mortality using direct links to the Office of National Statistics (where all deaths in England are registered). It started collecting data in 2000, so does not have comprehensive data for comparison with all the epochs described in Aylin’s paper, but has data on 2,913 infants who had open-heart operations in England during the three year calendar period of 2000-2002. Aylin et al report only 2,607 infant operations in their three year epoch 6 (1999-2002), suggesting serious errors in their case ascertainment. They report an overall English perioperative mortality for infant open-heart operations in epoch 6 at 4%, with 105 deaths identified over the three year period. The CCAD has identified 185 deaths (7.8% mortality) in the cohort of 2,385 open- heart infant operations during 2000-2002. CCAD’s validated, centre specific mortality for all open-heart infant operations in England for 2000-2002 ranged between 3.3% and 10.7%. Two centres had higher mortality than Oxford in this period, in stark contrast to Aylin’s report. The 95% confidence intervals for difference in mortality between Oxford and all of England over this period were –4.1% to +8.7%: a large overlap which means any difference is statistically insignificant. 1. Aylin et al. Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991- 2002. BMJ 2004;329:825-7. 2. Aylin P, Alves B, Best N et al. Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984-1996: was Bristol an Outlier? Lancet 2001;358:181-7. 3. Gibbs J L, Monro J L, Cunningham D, Rickards A. Survival after heart surgery or therapeutic catheterisation for congenital heart disease in the United Kingdom: analysis of the central cardiac audit database. BMJ 2004;328:611-5. Competing interests: None declared |
|||
|
|
|||
|
Paul Aylin, Clinical Senior Lecturer in Epidemiology and Public Health Dept. Epidemiology and Public Health, St. Mary's Campus, Imperial College. London W2 1PG, Brian Jarman, Alex Bottle
Send response to journal:
|
Sir, We note with interest that in the correspondence following our paper there is no disagreement with our findings of a marked decrease in mortality at Bristol and a more general reduction in national mortality over the past ten years. Criticism focuses on our publication of mortality by centre and in particular, our methods for risk adjustment and the validity of HES data for monitoring mortality. Parry et al. call for clarity of our methods for risk adjustment. We adjusted by type of operation, by incorporating the 11 open procedure groups as factors into our regression model. The use of procedure groups for risk adjustment is in line with other published methods of risk adjustment.[1] We also analysed those aged under 1 separately from those aged 1-15. In contrast, centre comparisons recently published by CCAD made no attempt at risk adjustment either by age or procedure group. [2] The method of analysis used in PICANet would be inappropriate to examine outcomes in paediatric cardiac surgery as a) patients who die on the table would be excluded and b) as illness severity is assessed on admission to intensive care units, risk adjustment is made on the post-operative state. Gibbs et al. remain concerned by the lack of accuracy of Hospital Episode Statistics. We agree that data quality is an important issue and there will inevitably be inaccuracies within our data, as in any clinical database. However, extensive comparisons with other data sources commissioned by the Inquiry showed reasonable agreement between them. In particular, HES seemed to record 99% of 30-day postoperative deaths in hospital for the procedures of interest.[3] HES is also the only database spanning our period of analysis from 1991 to 2002. In this complex arena of research, there will inevitably be differences between our figures and those included in Gibbs et al’s correspondence. However the two analyses are not necessarily inconsistent. Firstly, we point out that in their letter, Gibbs et al confusingly give two different figures (2,913 and 2,385) for the number of infants who had open-heart operations in England during the three year calendar period of 2000 to 2002. Secondly, we only looked at in-hospital mortality and the results published from CCAD includes all peri-operative deaths. Thirdly, there is nothing within HES to specify whether an operation is open and so this must be inferred from the operation type. To be consistent with the earlier analysis, the same list of procedures was used to determine whether an operation was open or closed.[4] This may partly explain the apparent difference in open operations within HES. However, we also examined mortality in 11 well defined open procedure groups which gave similar differences between centres. Lastly, Gibbs et al provide CCAD results for a different time period. Oxford have already confirmed to us that because of a number of deaths occurring in late 1999 (hence not included in the CCAD figures but included in our figures for epoch 6) and early 2000, they stopped performing TGA surgery from May 2000 (J Morris, medical director, Oxford Radcliffe Hospitals NHS Trust, personal communication, 2 April 2004). We welcome the work of the CCAD in monitoring the quality of outcomes for paediatric surgery patients in the UK and see it is a valuable source of information. We would be delighted to see centre-based comparisons based on the most recent three years of CCAD data, particularly as Gibbs et al are keen to suggest that there are two centres with higher mortality than Oxford. Whatever criticisms are made of HES, it is collected independently of clinicians and is available for public scrutiny. Further collaborative work to identify and correct inconsistencies between HES and clinical data sets might be a useful consequence of our work and could enhance the credibility of both sources of data. 1. Consensus-based method for risk adjustment for surgery for congenital heart disease Jenkins K, Gauvreau K, Newburger J, Spray T, Moller J, Iezzoni L. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123:110-118 2. Gibbs J, Monro J, Cunningham D, Rickards A. Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000-1. BMJ 2004;328: 611-20. 3. Aylin P, Alves B, Best N, Cook A, Elliott P, Evans SJW, et al. Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984-96: was Bristol an outlier? Lancet 2001;358: 181-87. 4. Aylin P, Alves B, Cook A, Bennett J, Bottle A, Best N, Catena B, Elliott P. Analysis of hospital episode statistics for the Bristol Royal Infirmary inquiry. London: Division Primary Care and Population Health Sciences, Imperial College London, 1999. www.bristol- inquiry.org.uk/Documents/hes_(Aylin).pdf Competing interests: The work was funded by Dr Foster Limited. BJ served on the panel for the Bristol Royal Infirmary inquiry. PA was an expert witness for the inquiry. |
|||
|
|
|||
|
James Morris, Medical Director John Radcliffe Hospital, Oxford OX3 9DU, Trevor Campbell Davis (Chief Executive), Ravi Pillai, Stephen Westaby (Consultant Cardiac Surgeons), Satish Adwani, Nick Archer, Neil Wilson (Consultant Paediatric Cardiologists)
Send response to journal:
|
Editor We write to express our concern that the electronic responses to Aylin et al (1) have not yet been published in the paper edition of the BMJ. One of the striking points made in this paper was that, on the basis of hospital episode statistics (HES) data, Oxford infant open cardiac surgical mortality through the 1990s and into 2002 was significantly greater than in the other English centres for which information was given. Clearly this assertion is a powerful and potentially sensational one, whether or not it is in fact true. A reply from Oxford was posted on the BMJ rapid response website promptly. Somewhat later, strong criticism of Aylin's data came in a joint response from representatives of the British Paediatric Cardiac Association, the Society of Cardiothoracic Surgeons and the Central Cardiac Audit Database (CCAD) and was posted on the website. As yet, these criticisms of the data and of the conclusions have not been made available to the wider BMJ readership who are unlikely to have followed the rapid responses. It is not surprising that this paper caused widespread upset to patients and professionals and that it was followed by adverse publicity in the popular press. We look forward to publication of the various responses to the original paper, so that at least the readership in general will know that there are powerful arguments against the reliability of HES data. Reliable validated data are available from CCAD for the latter part of the time described by Aylin which do not show Oxford as an outlier. There is no reason to suppose that HES data were more accurate before 2000, indeed the original paper notes that the quality of HES data ascertainment improved throughout the time covered. It is our view that, whether or not the conclusions drawn about our centre by Aylin and colleagues are valid (and we believe that they are not), this was not the best way to address them. However, having published them, we believe that it is essential for the BMJ to publish now in print the objections to the data not only from ourselves but, more importantly, from external and objective authorities. If the BMJ wishes to maintain its standing as a serious and respected professional journal, it owes this to the many families and professionals distressed by the publication of such an unvalidated, controversial and damaging report. 1. Aylin P, Bottle A, Jarman B. Paediatric Cardiac Surgical Mortality in England after Bristol:descriptive analysis of hospital episode statistics 1991-2000 BMJ 2004;329:825-27 (9 October) Competing interests: All the authors work for Oxford Radcliffe Hospitals NHS Trust |
|||