Economic evaluation and randomised controlled trial of extracorporeal membrane oxygenation: UK collaborative trialCommentary: Concurrent economic evaluations are rare but should be standard practice
BMJ 1998; 317 doi: https://doi.org/10.1136/bmj.317.7163.911 (Published 03 October 1998) Cite this as: BMJ 1998;317:911Abstract
Objective: To compare the resource implications and short term outcomes of extracorporeal membrane oxygenation and conventional management for term babies with severe respiratory failure.
Design: Cost effectiveness evaluation alongside a randomised controlled trial.
Setting: 55 approved recruiting hospitals in the United Kingdom. These hospitals provided conventional management, but infants randomised to extracorporeal membrane oxygenation were transferred to one of five specialist centres.
Subjects: 185 mature newborn infants (gestational age at birth >35 weeks, birth weight >2 kg) with severe respiratory failure (oxygenation index >40) recruited between 1993 and 1995. The commonest diagnoses were persistent pulmonary hypertension due to meconium aspiration, congenital diaphragmatic hernia, isolated persistent fetal circulation, sepsis, and idiopathic respiratory distress syndrome.
Main outcome measure: Cost effectiveness based on survival at 1 year of age without severe disability.
Results: 63 (68%) of the 93 infants randomised to extracorporeal membrane oxygenation survived to 1 year compared with 38 (41%) of the 92 infants who received conventional management. Of those that survived, one infant in each arm was lost to follow up and the proportion with disability at 1 year was similar in the two arms of the trial. One child in each arm had severe disability. The estimated additional cost of extracorporeal membrane oxygenation per additional surviving infant without severe disability was £51 222 and the cost per surviving infant with no disability was £75 327.
Conclusions: Extracorporeal membrane oxygenation for term neonates with severe respiratory failure would increase overall survival without disability. Although the policy will increase costs of neonatal health care, it is likely to be as cost effective as other life extending technologies.
Economic evaluation and randomised controlled trial of extracorporeal membrane oxygenation: UK collaborative trial
- Tracy E Roberts, lecturer in health economics (robertte{at}hsmc.bham.ac.uk)
- the Extracorporeal Membrane Oxygenation Economics Working Group on behalf of the Extracorporeal Membrane Oxygenation Trial Steering Group
- Health Economics Facility, University of Birmingham, Birmingham B15 2RT
- Royal Defence Medical College and Ministry of Defence, Ash Vale, Hants GU12 5RR
- Accepted 16 June 1998
Abstract
Objective: To compare the resource implications and short term outcomes of extracorporeal membrane oxygenation and conventional management for term babies with severe respiratory failure.
Design: Cost effectiveness evaluation alongside a randomised controlled trial.
Setting: 55 approved recruiting hospitals in the United Kingdom. These hospitals provided conventional management, but infants randomised to extracorporeal membrane oxygenation were transferred to one of five specialist centres.
Subjects: 185 mature newborn infants (gestational age at birth >35 weeks, birth weight >2 kg) with severe respiratory failure (oxygenation index >40) recruited between 1993 and 1995. The commonest diagnoses were persistent pulmonary hypertension due to meconium aspiration, congenital diaphragmatic hernia, isolated persistent fetal circulation, sepsis, and idiopathic respiratory distress syndrome.
Main outcome measure: Cost effectiveness based on survival at 1 year of age without severe disability.
Results: 63 (68%) of the 93 infants randomised to extracorporeal membrane oxygenation survived to 1 year compared with 38 (41%) of the 92 infants who received conventional management. Of those that survived, one infant in each arm was lost to follow up and the proportion with disability at 1 year was similar in the two arms of the trial. One child in each arm had severe disability. The estimated additional cost of extracorporeal membrane oxygenation per additional surviving infant without severe disability was £51 222 and the cost per surviving infant with no disability was £75 327.
Conclusions: Extracorporeal membrane oxygenation for term neonates with severe respiratory failure would increase overall survival without disability. Although the policy will increase costs of neonatal health care, it is likely to be as cost effective as other life extending technologies.
Key messages
Extracorporeal membrane oxygenation increases survival for term neonates in respiratory failure
The technique was three times more costly than conventional management
If extracorporeal membrane oxygenation is adopted it will increase the cost of neonatal health care
Extracorporeal membrane oxygenation may be as cost effective as other life extending technologies, but long term follow up studies are needed to confirm this
Introduction
Between 100 and 200 mature newborn infants die each year in the United Kingdom because of severe respiratory failure.1 Conventional treatment for infants in respiratory failure is ventilation with high level oxygen. Extracorporeal membrane oxygenation is a technique which oxygenates blood outside the body, obviating the need for gas exchange in the lungs, and, if necessary, providing cardiovascular support. As an expensive new technology in limited use, it was suitable for evaluation under the United Kingdom's health technology assessment model in the NHS research initiative.2 The UK collaborative randomised extracorporeal membrane oxygenation trial included an economic evaluation as an integral part of the design. Intention to treat analysis showed that extracorporeal membrane oxygenation was highly clinically effective. 34
A preliminary economic evaluation carried out before the trial suggested that extracorporeal membrane oxygenation was probably more effective and more expensive than conventional management.5 But it also showed that the existing evidence on cost effectiveness was inadequate for setting priorities because the uncertainty surrounding the data was too great.6 We report here the economic evaluation of the trial.
Subjects and methods
The economic evaluation was a cost effectiveness analysis comparing extracorporeal membrane oxygenation with conventional management based primarily on the principal clinical outcome of the trial (survival without severe disability at age 1 year). The trial protocol was developed in collaboration with health economists, who were members of the trial steering group, and an economics working party oversaw the economic evaluation. The economic evaluation was conducted from the viewpoint of the NHS and so includes only direct costs to the health service. Data on the costs to the parents and families of the infants will be discussed elsewhere.
Extracorporeal membrane oxygenation was provided in five centres, and babies were recruited from 55 UK neonatal centres. Babies were eligible for the trial if they were mature newborn infants with severe respiratory failure. They were randomised either to be transferred for extracorporeal membrane oxygenation or to receive conventional management.4 Babies in both arms of the trial received some of their care in neonatal units.
The trial data set included indicators of use of key resources, which were costed by the economics researchers using a combination of methods. Health service use was divided into three components: mode of transport used for transfers made after randomisation until discharge; services received in the initial hospital inpatient stay after randomisation, subdivided by level of intensity; and use of health services from discharge up to 1 year of age.
The babies' initial hospital treatment was described in terms of five levels: days receiving extracorporeal membrane oxygenation; days receiving maximal intensive care (more than 90% oxygen); days on a ventilator (receiving less than 90% oxygen); days on supplementary oxygen; and days in normal care.
Costs
Secondary data for the costs of days not receiving extracorporeal membrane oxygenation were taken from a parallel study conducted by some of the same researchers (economic evaluation of surfactant (ECSURF) study). 78These data were likely to be more representative with less risk of bias than data collected in a few centres. 69
The suitability of these secondary data for our analysis was assessed in a pilot study before the trial.6 It concluded that data from the ECSURF study would be appropriate, although during the acute phase of their illness babies in this trial would require more drugs and investigations than those in the ECSURF study.6 Based on analysis of these additional resources, and taking into account the case mix of the units in the ECSURF study and the corresponding size of the neonatal units in this trial, a weighting factor of 10% was applied to the ECSURF data. 610
Full details of the ECSURF study methods, results, and final application to the results of this trial have been reported.11 The adjusted costs, estimated in 1991 prices, were inflated to 1994-5 prices by using the combined hospital and community health services index and weighted further by 10% to reflect the changes in junior hospital doctors' hours and training since the 1993 Calman report.12 final summary figure to represent the average cost for the spectrum of neonatal care received by babies for the days on which they were not receiving extracorporeal membrane oxygenation was produced for each arm of the trial.
The information used to calculate the average cost of an extracorporeal membrane oxygenation day was collected directly from the four centres which provided nearly all the extracorporeal membrane oxygenation for the infants in the trial. When possible the costs were collected by using a “top down” approach following the methods used in the ECSURF study and elsewhere.13This approach calculates the total cost of the neonatal unit and apportions this to different categories of patient. Each extracorporeal membrane oxygenation centre was sent a detailed questionnaire based on the ECSURF study survey form requesting cost data for the financial year April 1994-5. TER visited each centre to ensure consistency in the apportionment and reporting of cost data. The costs of lighting, heating, and buildings were excluded for both arms of the trial. These omitted costs are not likely to affect the estimated relative difference between the two arms, but the absolute costs estimates for both policies will be lower than can be expected in practice.
The costs to the health service of an infant death, which consisted mainly of the cost of a postmortem examination and additional transport, were calcu- lated for both arms of the trial and included in the overall cost of hospital care. However, data regarding the counselling received by parents after an infant death were not collected, and these costs are not included.
The costs of each ambulance journey were estimated by using the method of the London Ambulance Service (personal communication). The total ambulance cost for each trip was made up of three maincomponents: a fixed fee for the vehicle, a rate for mileage, and an hourly rate for the total time the ambulance was in use. A minimum charge per journey was applied if the total cost of any journey fell below the threshold.10 All the air transport costs were supplied by the relevant companies and services from their own records.
The preliminary economic evaluation before the trial established that the costs from discharge up to age 1 year would be a relatively minor proportion of the total costs.5 Data were collected about use of health services after discharge from hospital through to age 1, and we used published unit costs from the University of Kent.14 All primary cost data were estimated in 1994-5 prices. Secondary cost data were adjusted for inflation by using the combined hospital and community health services index.
We did a sensitivity analysis using a range of plausible assumptions. Changes in five key variables were considered and the resulting effect on the incremental cost effectiveness ratio, if any, was estimated:
(1) The daily cost of extracorporeal membrane oxygenation taken as that in the highest cost and lowest cost centres.
(2) A change in numbers receiving extracorporeal membrane oxygenation. Three scenarios were tested. Firstly, the average neonatal capacity for 1994-5 was doubled (that is, from 14% to 28%). This is plausible because during the trial half the eligible infants were randomised to conventional management. Secondly, the average occupancy for extracorporeal membrane oxygenation cots was set at 70%, the maximum plausible capacity to maintain bed availability for new admissions. Thirdly, the average use was set at 50%.
(3) A change in the mode of transport for some journeys, replacing air with road transport.
(4) A change in the difference in survival between the two arms of the trial for the principal outcome using the upper and lower limits of the confidence interval for attributable benefit (0.26; 95% confidence interval 0.13 to 0.40). The difference in survival determines the denominator of the cost effectiveness ratio.
(5) Changes in staffing levels. Trial conditions or the introduction of a new service may not be representative of how the service is developed when the costs are no longer driven by the trial protocol. However, a telephone survey after the trial showed that the staff resources allocated to each neonatal case in the extracorporeal membrane oxygenation centres had not fallen, and no further sensitivity analysis was carried out using this variable.
Results
Ninety three babies were randomised to receive extracorporeal membrane oxygenation and 92 to conventional management. At 1 year of age, extracorporeal membrane oxygenation had increased survival (63 v 37), the number of survivors without severe disability (61 v 36), and the number of survivors with no disability (49 v 32). However, it also increased the number of survivors with impairment or disability (17 v 10) (table 1
The inpatient stay in the extracorporeal membrane oxygenation arm exceeded that in the conventional management arm by 1767 days (table 2) because more infants receiving conventional management died and they died early. In all, 51% (19) of the survivors who received conventional management required readmission to hospital compared with 35% (22) who received extracorporeal membrane oxygenation.
The estimated average cost for days without extracorporeal membrane oxygenation in the conventional management arm was £230. The cost in the extracorporeal membrane oxygenation arm was £205. The last estimate is lower because the infants spent the more resource intensive phase of their care receiving extracorporeal membrane oxygenation.
The estimated average total cost per day of extracorporeal membrane oxygenation was £1813. The average occupancy of the units during the trial was 14%. We estimated that average cost could fall to £1534 with 70% occupancy of the cots (table 3). In all three scenarios the average cost per day falls because the opportunity cost of increasing equipment use, up to full capacity, is zero. Although staff are involved, it is unrealistic to assume that they will be idle when throughput is low.
Table 4 shows the breakdown of the total health service costs up to 1 year of age. The total days with and without extracorporeal membrane oxygenation in each arm of the trial were multiplied by their corresponding average cost per day and summed to calculate the total cost of the initial hospital stay for each arm. The Appendix shows the data used to calculate the total transport costs and costs from discharge until age 1 year. The total costs after discharge for the survivors who received extracorporeal membrane oxygenation is almost twice that of those who received conventional management because there were almost twice as many survivors. The total cost of extracorporeal membrane oxygenation was £1 936 824 and the total cost of conventional management was £644 180.
Extracorporeal membrane oxygenation increases survival as defined in terms of the two main outcomes of the trial but at additional cost. The additional cost of extracorporeal membrane oxygenation over conventional management for every additional survivor without severe disability at 1 year of age was estimated to be £51 222. The additional cost per additional survivor with or without impairment but with no disability at one year was £75 327 (table 5).
Discussion
Extracorporeal membrane oxygenation for babies with severe respiratory failure is more costly than conventional management, and if adopted it will increase the costs of neonatal health care. Since our results showed that it is also more clinically effective, health service decision makers will have to consider the cost effectiveness when deciding whether to introduce extracorporeal membrane oxygenation.
The additional cost per additional survivor without severe disability at age 1 year was about £51 222. This figure should be compared with the incremental costs of other life extending technologies. For neonates few other technologies have been properly evaluated in randomised controlled trials, and even when trials exist comparisons have to be made with caution because the figures are often derived by different methods. A randomised controlled trial of surfactant replacement treatment for severe neonatal respiratory distress syndrome conducted in 1990 found it was highly effective. The estimated cost was £18 604 (at 1994-5 prices) per additional survivor.15
A study which set out to devise a mechanism to provide carefully established recommendations on new technologies suggested that technologies with a cost utility of £3000 to £20 000 per (quality adjusted) life year gained should be strongly recommended if the evidence came from a properly designed randomised controlled trial.16Extracorporeal membrane oxygenation resulted in more survivors with impairment or disability at 1 year than conventional management but the proportions in the two arms were similar (27% v 31%). The long term costs and benefits of caring for this group cannot yet be estimated since it will depend on the extent to which the impairment or disability affects their development. The quality of life for all the trial survivors is being assessed in four and seven year follow up studies and the cost per quality adjusted life year (QALY) will be estimated then. If survivors are shown to have a near normal life expectancy and quality of life, the cost per QALY is likely to be at the low end of the recommended range. Because of the short term perspective of our analysis, we feel that the study supports adoption of extracorporeal membrane oxygenation, but this conclusion should be treated with caution until evidence about longer term effects emerges.
If it is accepted that extracorporeal membrane oxygenation is relatively cost effective, the NHS must consider whether the results can be generalised and decide how to organise the service. The capacity of the extracorporeal membrane oxygenation centres during the trial was underused, and so costs might be lower if services were further centralised. Higher occupancy would also make more efficient use of highly trained staff. Training costs for these staff were not obtained, but these may be important if a full service was developed. Transport costs could be reduced by almost half if centres were better placed.
Our trial provides a good example of an expensive technology being introduced in the context of a carefully planned evaluation. The integration of economics into the trial design established a multidisciplinary framework for collaboration which fostered a comprehensive approach to the research question and allowed the appropriate investigations to be done. The evidence that this collaboration has produced so far suggests that extracorporeal membrane oxygenation can be as cost effective as other life extending technologies regularly used in developed countries. However, until the results of the long term follow up studies become available, this conclusion should be viewed with caution.
Acknowledgments
Members of the Extracorporeal Membrane Oxygenation Economics Working Group were T Roberts, M Mugford, C Normand, D Elbourne, D Field, A Grant, C Harris, A Johnson, and A Wrotchford. Former members were S Howard (researcher for economic evaluation 1992-4), L Hallam (researcher for economic evaluation 1995), K Enock (trial administrator, 1992-6).
Contributors: TER was principal economic researcher responsible for the primary cost fieldwork, economic analyses, and writing the paper and is the study guarantor. M Mugford and C Normand were responsible for the design, funding application, and overseeing of the economic evaluation and edited drafts of the paper. D Elbourne, D Field, and A Grant secured the original trial funding, developed its design, and took part in the economic evaluation. D Field and A Johnson provided clinical input. C Harris was data manager and A Wrotchford was computer programmer and assisted in analysis of data output. All commented on drafts of the paper.
Members of the steering group were D Field (chairman), C Davis, D Elbourne (trial coordinator), A Grant (trial coordinator), A Greenough, P Hale, L Hamilton, A Johnson, M Levene, M Liddell, F Locket, D Macrae, and C Skeoch.
We thank the babies' parents, the medical and nursing staff at the extracorporeal membrane oxygenation centres for providing cost data and clarifying queries, the authors of the ECSURF study for providing cost data, and our colleagues at the National Perinatal Epidemiology Unit and elsewhere for their advice.
Footnotes
Funding Funding: England and Wales Department of Health, Chief Scientist's Office, and Scottish Office Department of Health.
Conflict of interest None
Commentary: Concurrent economic evaluations are rare but should be standard practice
- Tom Jefferson, Edmund Parkes professor of preventive medicine (Zorria{at}epinet.co.uk)
- Health Economics Facility, University of Birmingham, Birmingham B15 2RT
- Royal Defence Medical College and Ministry of Defence, Ash Vale, Hants GU12 5RR
Randomised trials are currently the most robust method of comparing effectiveness of new interventions such as extracorporeal membrane oxygenation with current practice. Assessment of effectiveness alone, however, is unlikely to determine whether the new technology makes best use of available resources. This is an important aspect to be assessed in the introduction of all new technologies but especially for this one, given the notable start up and capital cash costs of extracorporeal membrane oxygenation and the uncertainty surrounding the performance of the new technology. The question is answered by the economic evaluation carried out by Roberts and colleagues, which is published two years after the original trial. The two year publication gap partially conceals the occurrence of a rare event: both trial and economic evaluation were designed and carried out together, with economists and trialists working together from the earliest stages of the venture.
Economic evaluations carried out and published alongside clinical trials are still a rarity, probably occurring in less than 1% according to one survey of studies published between 1966 and 1988.1 In another study, only one economic evaluation of the 45 (2.2%) submitted to the BMJ and Lancet over nine months had been carried out alongside a clinical trial.2
The benefits are clear: the application of more than one perspective to answering a difficult problem (namely, whether we should introduce an expensive new technology) has given an answer. There is a further benefit, the demonstration that multidisciplinary approaches to problems do work. For example, study design and data collection are decided at protocol stage and more likely to be adequate for both assessments, resulting in better and more meaningful data. The taxpayer can rest assured that the bodies funding the extracorporeal membrane oxygenation trialists have acted in the public interest by commissioning a multidimensional study which answered the question of whether the technique was worth introducing. This contrasts with the standard approach of calling on economists' advice at the last minute before submitting a grant application in impossible “salvage” attempts.
Why, then, have the trial, the long term follow up results, and the economic evaluation, admirably part of the same study, been published separately in two journals over two years? The answer may lie in a variety of factors including publication space considerations, prejudice against economists, unwillingness to address economic issues at the protocol design stage, and uncertainty about the rationale and techniques of economic evaluation.3 Willingness to publish the results of the assessment of the clinical effects of a new intervention but not its impact on resources seems irrational, similar to an editorial salami slicing process. Whatever the reason, we should be grateful that both trial and economic evaluation are now published in international journals to serve as an example of things to come, I hope.
Acknowledgments
Dr Miranda Mugford checked the factual accuracy of this commentary