BMJ  2004;328:611 (13 March), doi:10.1136/bmj.38027.613403.F6 (published 24 February 2004)

Paper

Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000-1

¹ Central Cardiac Audit Database, Department of Paediatric Cardiology, Leeds General Infirmary, Calverley, Leeds LS1 3EX jgibbs{at}boltblue.com, ² Society of Cardiothoracic Surgeons of England and Ireland, Southampton General Hospital, Southampton SO16 6YD, ³ Central Cardiac Audit Database, Royal Brompton Hospital, London SW3 6NP

Abstract

Objectives To analyse simple national statistics and survival data collected in the central cardiac audit database after treatment for congenital heart disease and to provide long term comparative statistics for each contributing centre.

Design Prospective, longitudinal, observational, national cohort survival study.

Setting UK central cardiac audit database.

Main outcome measures Survival at 30 days and one year after treatment in the year April 2000-March 2001, assessed by using both volunteered life status and independently validated life status through the Office for National Statistics, using the patient's unique NHS number, or the general register offices of Scotland and Northern Ireland. Institutional results following a group of six benchmark operations and three benchmark catheterisation procedures.

Results Since April 2000 data have been received from all 13 UK tertiary centres performing cardiac surgery or therapeutic cardiac catheterisation in children with congenital heart disease. Altogether 3666 surgical procedures and 1828 therapeutic catheterisations were performed. Central tracking of mortality identified 469 deaths, 194 occurring within 30 days and 275 later. Forty two of the 194 deaths within 30 days were detected by central tracking but not by volunteered data. For surgery overall, survival at 30 days was 94.9%, falling to 91.2% at one year; this effect was most marked for infants. For therapeutic catheterisation survival at 30 days was 99.1%, falling to 98.1% at one year. Survival of individual centres or individual operators did not differ from the national average after benchmark procedures.

Conclusions Independent data validation is essential for accurate survival analysis. One year survival gives a more realistic view of outcome than traditional perioperative mortality. Currently no detectable difference exists in survival between any of the 13 UK tertiary congenital heart disease centres, but confidence intervals for small centres are wide, limiting our power to detect underperformance from analysis of a single year's data. Appropriately resourced, focused national audit is capable of accurate data collection on which nationwide, long term quality control can be based.

Introduction

Monitoring of survival rates after cardiac surgery was introduced in the United Kingdom in 1977 with voluntary submission of data to the Society of Cardiothoracic Surgeons of Great Britain and Ireland. The central cardiac audit database was established by the British Cardiac Society, the Society of Cardiothoracic Surgeons, and the British Paediatric Cardiac Association to provide national analysis of outcomes of cardiac surgery and therapeutic cardiac catheterisation. It differs in three major aspects from previous national audit projects: data are collected electronically in a secure format; mortality and reintervention are tracked centrally by using a unique patient identifier (the NHS number); and independent data validation is used. In 2000 the Department of Health funded the central cardiac audit database to collate data from all centres for congenital heart disease in the United Kingdom. This report contains the first year's data (1 April 2000 to 31 March 2001), with centrally tracked one year survival. The results are presented on behalf of the Society of Cardiothoracic Surgeons, the British Paediatric Cardiac Association, and all contributing centres, each of which gave consent to publication of identifiable, centre specific data.

Methods

Data collection
We designed a minimum dataset of 20 fields with the simple aims of the project in mind. All 13 congenital heart disease centres in England, Scotland, and Northern Ireland participated. To ensure patient confidentiality the central cardiac audit database employs advanced data encryption technology to control access to data through a secure key system. We used lists with fixed choices consisting of all but the rarest and most complex combinations of diagnoses and procedures to minimise the potential complexities of diagnostic and procedural coding for congenital heart disease.

Data validation
The minimum dataset included date of death, but we linked with the Office for National Statistics by using NHS numbers to assess mortality wherever possible. We compared volunteered mortality data with centrally tracked data. In Northern Ireland and Scotland we used the general register offices to track mortality centrally.

The central cardiac audit database includes other forms of independent data validation carried out by visiting centres, when two weeks' submitted data, chosen at random, are compared with hospitals' written medical records, with operating theatres' records, and with laboratory records on cardiac catheterisation. We checked entries in the logbooks for operating theatres and catheter laboratories for the entire year in each centre, to ensure complete ascertainment of caseload. We also compared submitted data with nationally held hospital episode statistics whenever these were accessible.

Data analysis
We used the online Lotus Domino version of the central cardiac audit database to collect data and transferred these for analysis to SPSS 10.0 for Microsoft Windows. We used Wilson's score method to calculate confidence intervals for survival.^{1 2} We used 99% confidence intervals (table) to make allowances for the high number of multiple comparisons, to minimise false positive results. We did not consider an individual survival value to be significantly different from the mean if the confidence intervals overlapped the mean.

View this table: [in this window] [in a new window]   Survival for neonates, infants, all children under 1 year, and children between 1 year and 16 years undergoing surgery or therapeutic catheterisation

 

We used analysis of survival after "benchmark" operations to compare results from different centres, to eliminate the effect of different case mix. We chose six benchmark procedures for surgery (repairs of atrial septal defect, ventricular septal defect, atrioventricular septal defect, tetralogy of Fallot, simple transposition of the great arteries, and coarctation) and three for therapeutic catheterisation (atrial septal defect closure, arterial duct closure, and pulmonary balloon valvoplasty). We did not undertake detailed risk stratification as no validated method exists.

We calculated 30 day postoperative survival to facilitate comparison with results from previous UK registry data and to comply with practice in the United States.³ Central tracking of mortality has, however, also allowed us to plot one year survival curves, in contrast to previous registry analyses. We included foreign nationals but censored them from survival analysis after the perioperative period unless specific follow up data were available (central tracking was not possible for this group). We analysed individual operators' results anonymously, but each centre agreed to be identified. We have not calculated freedom from reintervention statistics for this report as follow up is currently too short to allow meaningful interpretation of results.

Results

Data collection and quality
Overall completeness of the dataset was 96.8%, with completeness for individual data fields ranging from 75% (for NHS number) to 100%. Data were received for a total of 5494 procedures, of which 3666 were surgical and 1828 were therapeutic catheterisations.

We have reported all cause mortality, choosing not to attempt detailed investigation of cause of death and its relation to treatment. We found substantial differences in volunteered and centrally tracked mortality, with seven of 11 centres in England under-reporting death within 30 days. Central tracking of mortality identified 469 deaths, 194 occurring within 30 days and 275 later. Forty two of the 194 deaths within 30 days were detected by central tracking but were not in the volunteered data. Nineteen of these patients were discharged alive but subsequently died within 30 days of the operation. The remainder had been incorrectly coded as alive at discharge; using reported discharge status would have underestimated 30 day mortality by 22%. Data on hospital episode statistics were available for 2716 patients and under-reported death within 30 days by 9%, classifying 1% of surviving patients as deceased. Hospital episode statistics data also under-reported the total number of procedures by 10%. During validation visits we found a total of 143 procedures to be missing from the data submissions to the central cardiac audit database, predominantly related to systematic errors in data collection. The visits resulted in submission of missing or revised data from all of the 13 centres.

Survival
Figures 1, 2, 3 show national survival curves after cardiopulmonary bypass surgery, non-cardiopulmonary bypass surgery, or therapeutic catheterisation. The table shows survival at 30 days and one year after all procedures. We assessed benchmark procedure survival anonymously for individual operators (41 surgeons and 63 cardiologists) as well as for different centres. No significant difference from the national mean survival was detectable for any individual. Figures 4 and 5 and figures A-I (see bmj.com), respectively, show individual centres' survival data for pooled and individual benchmark procedures.

View larger version (22K): [in this window] [in a new window]   Fig 1 One year survival after cardiopulmonary bypass surgery for age groups <1 month, 1 month to 1 year, and 1 to 16 years for the United Kingdom

 

View larger version (22K): [in this window] [in a new window]   Fig 2 One year survival for non-cardiopulmonary bypass surgery for age groups <1 month, 1 month-1 year, and 1-16 years for the United Kingdom

 

View larger version (21K): [in this window] [in a new window]   Fig 3 One year survival for therapeutic catheterisation for age groups <1 month, 1 month to 1 year, and 1 to 16 years for the United Kingdom

 

View larger version (30K): [in this window] [in a new window]   Fig 4 Survival at 30 days and at one year reported by individual centre, with 99% confidence intervals for all benchmark surgical procedures. The shaded areas represent the national means with 99% confidence intervals. If a centre's confidence intervals overlap the shaded area, survival at the centre does not differ statistically from the national mean. For a list of the abbreviations see bmj.com

 

View larger version (30K): [in this window] [in a new window]   Fig 5 Survival at 30 days and at one year reported by individual centre, with 99% confidence intervals for all benchmark catheter procedures. The shaded areas represent the national means with 99% confidence intervals. If a centre's confidence intervals overlap the shaded area, survival at the centre does not differ statistically from the national mean. For a list of the abbreviations see bmj.com

 

Discussion

Data quality
We found a striking difference in deaths identified by centres' own records and by central tracking. This was most marked when death occurred after the perioperative period, but deaths were missing from submitted data even when death occurred within 30 days. It seems inevitable that previous registry reports (including the register of the Society of Cardiothoracic Surgeons, used in the Bristol inquiry) that have relied on voluntary reporting of death have also under-reported mortality, casting doubt on their validity. The introduction of the NHS number as a unique and permanent patient identifier and the ability to establish electronic linkage with the Office for National Statistics is a major advance in tracking patients' outcomes.

Risk stratification
The use of benchmark procedures minimises the effect of varying case mix for the purposes of comparison of outcomes in different centres. Attempts have been made to establish a consensus view of risk assessment in the United States⁴ and case complexity in the United Kingdom.⁵ These protocols, applied to the data in the central cardiac audit database that have been accumulated over several years, should facilitate development and validation of risk stratification for the future.

Patient confidentiality
We did not include patients' consent for data submission to the central cardiac audit database in our protocol. Our current understanding of the Data Protection Act 1998 is that patients' consent is not required if anonymised data are used for the purpose of research or audit. The exception to anonymising data is the NHS number, which we have protected by encryption with a key held only by the data managers and used only for record linkage. In the United Kingdom we have an almost unique opportunity to carry out effective and believable national audit because we have a single healthcare system with an Office for National Statistics where the life status of an individual patient, based on their NHS number, is known.

With appropriate precautions central tracking is possible while maintaining patient confidentiality.

Diagnostic and procedure coding
Several groups have devised coding systems for congenital heart disease.^{3 6 7} The central cardiac audit database plans to adopt the coding system of the Association for European Paediatric Cardiology⁷ from 2004, to facilitate future international compatibility of data.

Outcomes after treatment
Centre specific data analysis shows that quality of treatment is high throughout the United Kingdom; no centre and no individual operator has detectably different survival from the national mean after benchmark procedures. We have been unable to assess accurately how our results compare with those of other nations, although our data seem to compare favourably with unvalidated registry reports from North America and Europe.

What is already known on this topic The validity of previous voluntary registers of survival after surgery has long been held to be potentially inaccurate The Bristol inquiry report highlighted the inadequacy of current national audit, particularly for the treatment of congenital heart disease What this study adds Volunteered survival data are of little value, sometimes overestimating survival by as much as 20% Data validation is essential for national or local audit of survival and has been made far easier by the introduction of the NHS number and the ability to use it to create electronic links to the Office for National Statistics Traditionally reported perioperative (30 day) mortality can give a misleadingly optimistic view of prognosis to both professionals and the public; for infants mortality after treatment for heart disease at one year was double that at 30 days The central cardiac audit database places validated, centre specific survival results for treatment of children with congenital heart disease in the public domain

We believe that this study is the first to present validated, centre specific survival data for nationwide treatment of congenital heart disease. Population based, 45 year, actuarial survival has been reported for the whole of Finland,⁸ but individual centres' performance was not included. Most previous reports have concentrated on multicentre, mean perioperative mortalities (defined as death within 30 days of operation),⁹ and a similar approach was used for the register of the Society of Cardiothoracic Surgeons.¹⁰ Although this simplistic approach may be convenient, our data show how misleading 30 day results may be as a descriptor of overall outcome.

We calculated 99% confidence intervals for the purpose of assessing survival in different centres. Even at this level, having performed a total of 178 comparisons, we think that it is likely that we will generate spuriously significant results: we calculate that the chance of at least one spuriously significant difference in survival is 83% (this would have been 99.99% had we used 95% confidence intervals).

This early data analysis has concentrated on survival, which is a crude indicator of overall performance. For smaller centres, as well as for individual operators, analysis of a single year's data has limited power to detect underperformance by institutions or individual operators and year on year analysis will be necessary to provide more robust reassurance. Freedom from reintervention is likely to be a powerful indicator of overall performance, but several years' data will be required before our capability to track reintervention can be put to use.

Conclusions

Independent validation of data is essential for accurate survival analysis. One year survival statistics give a more realistic view of outcome than traditional perioperative mortality. At present survival is no different between any of the 13 UK tertiary centres for congenital heart disease, but confidence intervals are wide, limiting our power to detect underperformance from analysis of a single year's data. Appropriately resourced, focused, national audit is capable of accurate data collection on which nationwide, long term, quality control can be based.

---

Editorial by Treasure

Figures A-I and abbreviations are on bmj.com

This article was posted on bmj.com on 24 February 2004: http://bmj.com/cgi/doi/10.1136/bmj.38027.613403.F6

The paper was written on behalf of the Society of Cardiothoracic Surgeons of Great Britain and Northern Ireland, the British Paediatric Cardiac Association, and the congenital heart disease units of Alder Hey Hospital, Liverpool; Birmingham Children's Hospital; Bristol Royal Hospital for Sick Children; Freeman Hospital, Newcastle; Glenfield Hospital, Leicester; Great Ormond Street Hospital, London; Guy's Hospital, London; John Radcliffe Hospital, Oxford; Leeds General Infirmary; Royal Brompton and Harefield NHS Trust, London; Royal Hospital for Sick Children, Glasgow; Royal Victoria Hospitals, Belfast; and Southampton General Hospital.

Contributors: JLG, DC, and AR initiated the study. DC analysed the data. JLG, JLM, and AR drafted the report. All four authors are guarantors on behalf of the 13 centres that contributed data.

Funding: The National Health Service Information Authority (NHSIA) funded the central data collection, data validation, and data analysis.

Competing interests: None declared.

References

1. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 1927;22: 209-12.[CrossRef][ISI]
2. Newcombe R. Two sided confidence intervals for the single proportion: a comparative evaluation of seven methods. Stat Med 1998;17: 857-72.[CrossRef][ISI][Medline]
3. Mavroudis C, Jacobs JP. Congenital heart surgery nomenclature and database project: Overview and minimum dataset. Ann Thorac Surg 2000;69: S2-17.[Abstract/Free Full Text]
4. Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123: 110-18.[Abstract/Free Full Text]
5. Gallivan S, Davis KB, Stark JF. Early identification of divergent performance in congenital cardiac surgery. Eur J Cardiothorac Surg 2001;20: 1214-9.[Abstract/Free Full Text]
6. Lacour-Gayet F, Maruszewski B, Mavroudis C, Jacobs JP, Elliott MJ. Presentation of the international nomenclature for congenital heart surgery. The long way from nomenclature to collection of validated data at the EACTS. Eur J Cardiothorac Surg 2000;18: 128-35.[Abstract/Free Full Text]
7. Franklin RC, Anderson RH, Daniels O, Elliott M, Gewillig MH, Ghisla R, et al. Report of the coding committee of the Association for European Paediatric Cardiology. Cardiol Young. 2000;10(suppl 1): 1-26.
8. Nieminen HP, Jokinen EV, Sairanen HI. Late results of pediatric cardiac surgery in Finland: a population-based study with 96% follow-up. Circulation 2001;104: 570-5.[Abstract/Free Full Text]
9. Stark J, Gallivan S, Lovegrove J, Hamilton JR, Monro JL, Pollock JC, et al. Mortality rates after surgery for congenital heart defects in children and surgeons' performance. Lancet 2000;355: 1004-7.[CrossRef][ISI][Medline]
10. Aylin P, Alves B, Best N, Cook A, Elliott P, Evans SWJ, et al. Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984-1996: was Bristol an outlier? Lancet 2001;358: 181-7.[CrossRef][ISI][Medline]

(Accepted 2 December 2003)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Comparison of hospital episode statistics and central cardiac audit database in public reporting of congenital heart surgery mortality

Stephen Westaby, Nicholas Archer, Nicola Manning, Satish Adwani, Catherine Grebenik, Oliver Ormerod, Ravi Pillai, and Neil Wilson
BMJ 2007 335: 759. [Abstract] [Full Text] [PDF]

Paediatric cardiac surgical mortality after Bristol: Paediatric cardiac hospital episode statistics are unreliable

John L Gibbs, David Cunningham, Marc de Leval, James Monro, and Bruce Keogh
BMJ 2005 330: 43-44. [Extract] [Full Text]

Paediatric cardiac surgical mortality after Bristol: Authors' reply

Paul Aylin, Brian Jarman, and Paul Elliott
BMJ 2005 330: 44. [Extract] [Full Text]

Children treated for heart conditions survive equally well across UK
BMJ 2004 328: 0. [Full Text]

Congenital heart disease

Tom Treasure
BMJ 2004 328: 594-595. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Billett, J, Majeed, A, Gatzoulis, M, Cowie, M (2008). Trends in hospital admissions, in-hospital case fatality and population mortality from congenital heart disease in England, 1994 to 2004. Heart 94: 342-348 [Abstract] [Full text]  
• Westaby, S., Archer, N., Manning, N., Adwani, S., Grebenik, C., Ormerod, O., Pillai, R., Wilson, N. (2007). Comparison of hospital episode statistics and central cardiac audit database in public reporting of congenital heart surgery mortality. BMJ 335: 759-759 [Abstract] [Full text]  
• Jacobs, J. P., Jacobs, M. L., Mavroudis, C., Maruszewski, B., Tchervenkov, C. I., Lacour-Gayet, F. G., Clarke, D. R., Yeh, T. Jr, Walters, H. L. III, Kurosawa, H., Stellin, G., Ebels, T., Elliott, M. J., Vener, D. F., Barach, P., Benavidez, O. J., Bacha, E. A. (2007). What is Operative Morbidity? Defining Complications in a Surgical Registry Database. Ann. Thorac. Surg. 84: 1416-1421 [Full text]  
• Holm, I., Fredriksen, P. M., Fosdahl, M. A., Olstad, M., Vollestad, N. (2007). Impaired Motor Competence in School-aged Children With Complex Congenital Heart Disease. Arch Pediatr Adolesc Med 161: 945-950 [Abstract] [Full text]  
• Shinebourne, E. A, Babu-Narayan, S. V, Carvalho, J. S (2006). Tetralogy of Fallot: from fetus to adult.. Heart 92: 1353-1359 [Full text]  
• Brown, K L, Ridout, D A, Hoskote, A, Verhulst, L, Ricci, M, Bull, C (2006). Delayed diagnosis of congenital heart disease worsens preoperative condition and outcome of surgery in neonates. Heart 92: 1298-1302 [Abstract] [Full text]  
• Kang, N., Tsang, V. T., Gallivan, S., Sherlaw-Johnson, C., Cole, T. J., Elliott, M. J., de Leval, M. R. (2006). Quality assurance in congenital heart surgery.. Eur. J. Cardiothorac. Surg. 29: 693-697 [Abstract] [Full text]  
• Jacobs, J. P., Mavroudis, C., Jacobs, M. L., Maruszewski, B., Tchervenkov, C. I., Lacour-Gayet, F. G., Clarke, D. R., Yeh, T. Jr, Walters, H. L. III, Kurosawa, H., Stellin, G., Ebels, T., Elliott, M. J. (2006). What is Operative Mortality? Defining Death in a Surgical Registry Database: A Report of the STS Congenital Database Taskforce and the Joint EACTS-STS Congenital Database Committee. Ann. Thorac. Surg. 81: 1937-1941 [Abstract] [Full text]  
• Maruszewski, B., Lacour-Gayet, F., Monro, J. L., Keogh, B. E., Tobota, Z., Kansy, A. (2005). An attempt at data verification in the EACTS Congenital Database. Eur. J. Cardiothorac. Surg. 28: 400-404 [Abstract] [Full text]  
• Jacobs, J. P., Lacour-Gayet, F. G., Jacobs, M. L., Clarke, D. R., Tchervenkov, C. I., Gaynor, J. W., Spray, T. L., Maruszewski, B., Stellin, G., Gould, J., Dokholyan, R. S., Peterson, E. D., Elliott, M. J., Mavroudis, C. (2005). Initial Application in The STS Congenital Database of Complexity Adjustment to Evaluate Surgical Case Mix and Results. Ann. Thorac. Surg. 79: 1635-1649 [Abstract] [Full text]  
• Keogh, B. (2005). Surgery for congenital heart conditions in Oxford. BMJ 330: 319-320 [Full text]  
• White, C. (2005). Cardiac mortality in children in Oxford hospital is not excessive. BMJ 330: 324-324 [Full text]  
• Monro, J (2005). The changing state of surgery for adult congenital heart disease. Heart 91: 139-140 [Abstract] [Full text]  
• Aylin, P., Jarman, B., Elliott, P. (2005). Paediatric cardiac surgical mortality after Bristol: Authors' reply. BMJ 330: 44-44 [Full text]  
• Gibbs, J. L, Cunningham, D., de Leval, M., Monro, J., Keogh, B. (2005). Paediatric cardiac surgical mortality after Bristol: Paediatric cardiac hospital episode statistics are unreliable. BMJ 330: 43-44 [Full text]  
• Monro, J. L. (2004). The next challenge--adapting to change. Eur. J. Cardiothorac. Surg. 26: 1063-1072 [Full text]  
• Aylin, P., Bottle, A., Jarman, B., Elliott, P. (2004). Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991-2002. BMJ 329: 825- [Abstract] [Full text]  
• Weston, C F M (2004). Pre-hospital resuscitation: breathing life into a stale subject. Heart 90: 1107-1109 [Full text]  
• (2004). Lucina. Arch. Dis. Child. 89: 986-986 [Full text]  
• Treasure, T. (2004). Congenital heart disease. BMJ 328: 594-595 [Full text]  

Rapid Responses:

Read all Rapid Responses

Type II error in analysis of CCAD children's cardiac surgery survival?

Anthony P Roberts
bmj.com, 18 Mar 2004 [Full text]

Howler in "This Week in the BMJ"

Hugh Tunstall-Pedoe
bmj.com, 15 Mar 2004 [Full text]

Survival after Surgery or theraputic catherisation for congenital heart disease

Fiona M Woollard
bmj.com, 17 Mar 2004 [Full text]

Adjustment for procedure mix

Paul Aylin
bmj.com, 6 Apr 2004 [Full text]