Cost effectiveness of telehealth for patients with long term conditions (Whole Systems Demonstrator telehealth questionnaire study): nested economic evaluation in a pragmatic, cluster randomised controlled trialBMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f1035 (Published 22 March 2013) Cite this as: BMJ 2013;346:f1035
- Catherine Henderson, research officer1,
- Martin Knapp, professor of social policy, director of personal social services research unit12,
- José-Luis Fernández, deputy director of personal social services research unit, principal research fellow1,
- Jennifer Beecham, professorial research fellow1,
- Shashivadan P Hirani, senior lecturer in health services research3,
- Martin Cartwright, research associate in health services research3,
- Lorna Rixon, research associate in health services research3,
- Michelle Beynon, research assistant in health services research3,
- Anne Rogers, professor of health systems implementation 4,
- Peter Bower, professor of health services research5,
- Helen Doll, senior research associate6,
- Ray Fitzpatrick, professor of public health and primary care7,
- Adam Steventon, senior research analyst8,
- Martin Bardsley, head of research8,
- Jane Hendy, senior lecturer in healthcare management9,
- Stanton P Newman, dean, professor, principal investigator 3
- for the Whole System Demonstrator evaluation team
- 1London School of Economics and Political Science, London WC2A 2AE, UK
- 2King’s College London, London, UK
- 3School of Health Sciences, City University London, London, UK
- 4University of Southampton, Southampton, UK
- 5University of Manchester, Manchester, UK
- 6University of East Anglia, Norwich, UK
- 7University of Oxford, Oxford, UK
- 8The Nuffield Trust, London, UK
- 9University of Surrey, Guildford, UK
- Correspondence to: C Henderson
- Accepted 16 October 2012
Objective To examine the costs and cost effectiveness of telehealth in addition to standard support and treatment, compared with standard support and treatment.
Design Economic evaluation nested in a pragmatic, cluster randomised controlled trial.
Setting Community based telehealth intervention in three local authority areas in England.
Participants 3230 people with a long term condition (heart failure, chronic obstructive pulmonary disease, or diabetes) were recruited into the Whole Systems Demonstrator telehealth trial between May 2008 and December 2009. Of participants taking part in the Whole Systems Demonstrator telehealth questionnaire study examining acceptability, effectiveness, and cost effectiveness, 845 were randomised to telehealth and 728 to usual care.
Interventions Intervention participants received a package of telehealth equipment and monitoring services for 12 months, in addition to the standard health and social care services available in their area. Controls received usual health and social care.
Main outcome measure Primary outcome for the cost effectiveness analysis was incremental cost per quality adjusted life year (QALY) gained.
Results We undertook net benefit analyses of costs and outcomes for 965 patients (534 receiving telehealth; 431 usual care). The adjusted mean difference in QALY gain between groups at 12 months was 0.012. Total health and social care costs (including direct costs of the intervention) for the three months before 12 month interview were £1390 (€1610; $2150) and £1596 for the usual care and telehealth groups, respectively. Cost effectiveness acceptability curves were generated to examine decision uncertainty in the analysis surrounding the value of the cost effectiveness threshold. The incremental cost per QALY of telehealth when added to usual care was £92 000. With this amount, the probability of cost effectiveness was low (11% at willingness to pay threshold of £30 000; >50% only if the threshold exceeded about £90 000). In sensitivity analyses, telehealth costs remained slightly (non-significantly) higher than usual care costs, even after assuming that equipment prices fell by 80% or telehealth services operated at maximum capacity. However, the most optimistic scenario (combining reduced equipment prices with maximum operating capacity) eliminated this group difference (cost effectiveness ratio £12 000 per QALY).
Conclusions The QALY gain by patients using telehealth in addition to usual care was similar to that by patients receiving usual care only, and total costs associated with the telehealth intervention were higher. Telehealth does not seem to be a cost effective addition to standard support and treatment.
Trial registration ISRCTN43002091.
Management of people with long term conditions is under the spotlight, given the rapidly growing prevalence of such conditions in ageing populations. Treatment costs for people with long term conditions make up 69% of all spending on health and social care in England, and the number of people with at least one long term condition could rise by three million to 18 million by 2025.1 Among other factors, the rising financial impact of such conditions has been linked to increases in the lifetime prevalence of diabetes2 and other chronic diseases such as chronic obstructive pulmonary disease3 and congestive heart failure.4
There are compelling arguments for tackling long term conditions to improve quality of life, while being mindful of the need to contain costs.5 In particular, there is considerable interest in the potential of technologies such as telehealth to generate cost effectiveness gains and even to yield cost savings6 while maintaining or improving patient outcomes.6 7
Evidence on the effectiveness of telehealth is accumulating; systematic reviewers have judged it as promising for managing respiratory and cardiac disease and diabetes,8 9 10 11 despite study limitations such as small sample sizes and poor designs. Evidence of the effect of telehealth on service use, costs, or cost effectiveness remains scarce12 13 14 15 16 Furthermore, most studies have been conducted in the United States, leaving open the question of their relevance in the United Kingdom.14
The term “telehealth” encompasses both “telemonitoring” and telephone support. With telemonitoring, patients transmit data on their vital signs for real time monitoring, for instance, via video link or by store and forward systems (where data are submitted by the patient and transmitted to the health professional for later assessment).8 12 With telephone support, healthcare providers provide support to patients or carers via the standard telephone system. Telephone support can also involve monitoring of vital signs data as reported by patients.8 9 In practice, telehealth can be a hybrid of these approaches. We compared the costs and cost effectiveness of telehealth services (as an addition to standard support and treatment) with those of standard support and treatment alone. This analysis was part of the Whole System Demonstrator (WSD) programme, funded by the Department of Health.7 17
WSD telehealth trial
A pragmatic, cluster randomised controlled trial (the WSD telehealth trial) used routinely collected administrative datasets to examine the effect of telehealth on primary and secondary healthcare and social care use by individuals with long term conditions (chronic obstructive pulmonary disease, heart failure, or diabetes) in three demographically diverse sites.18 The WSD sites (local authority areas in England) were chosen for having an established record of joint working between health and social care; they were covered by four primary care trusts.
Randomisation took place at the general practice level. All practices within the four primary care trusts were eligible to participate. A minimisation procedure was devised to randomise practices to usual care or to a telehealth intervention in addition to standard care, to achieve a balance across trial arms of selected practice characteristics that were potentially associated with trial outcomes. These characteristics included site; practice size; index of multiple deprivation 2007 (IMD)19; proportion of white or non-white patients; and prevalence of diabetes, heart failure, and chronic obstructive pulmonary disease.
The patients at participating practices were eligible for study inclusion if they were 18 years or older, with at least one of three long term index conditions (chronic obstructive pulmonary disease, heart failure, or diabetes). Index conditions were confirmed by primary or secondary care medical records, the Quality Outcomes Framework register, or a local clinician. Patients who met diagnostic criteria but had cognitive impairments that would have prevented them from using the telehealth equipment independently could be recruited into the trial, provided that they had a carer who could assist them in using telehealth as prescribed in the context of the trial. We excluded non-English speaking patients from the trial, because the telehealth equipment provided information and instructions only in English.20
All patients that appeared to meet the study criteria, based on a review of practice records and input from local healthcare professionals, were invited to participate in the main trial. Patients who consented to sharing health and social care data with the research team were visited by the local WSD project team to confirm the patient’s eligibility and check the suitability of the patient’s home for telehealth. Eligible patients were presented with verbal and written information about the trial, and if they were willing to participate in the trial, were asked to provide informed written consent. The main telehealth trial recruited 3230 participants from 179 general practices between May 2008 and December 2009. Patients in the control group received usual health and social care for the 12 month follow-up period, and were then offered telehealth subject to a further clinical assessment. Patients in the intervention group received telehealth in addition to the standard health or social care services available in their area.20
WSD questionnaire study
The WSD telehealth questionnaire study was nested within the parent trial described above. The questionnaire study used participant reported outcomes collected from a subset (n=1573) of the parent trial sample and served as the basis for evaluating acceptability, effectiveness, and cost effectiveness of telehealth as a supplement to standard care.18 20 Patients who had cognitive impairments that would impair their ability to independently complete evaluation instruments were not eligible for the questionnaire study, although they were eligible for the parent trial, because data for outcome measures were to be collected first hand.20
All eligible patients were asked during the assessment visit by the WSD project team whether they would be willing to participate in the questionnaire study in addition to the main trial. Baseline interviews were arranged with those trial participants who agreed, and recruitment into the questionnaire study continued with the aim of reaching the 550 participants for each index condition (or 1650 in total).20
At baseline, patients agreeing to participate were visited by trained interviewers18 who obtained additional informed written consent for the questionnaire study, and collected information on primary and secondary outcomes. Data for service use were also collected using the Client Services Receipt Inventory.21 This instrument collects comprehensive information on patient service use, living arrangements, and employment status as well as patterns of unpaid care and support by their family or other carers. The Client Services Receipt Inventory and other study instruments were posted to participants at four and 12 months after randomisation. Patients who had not returned their questionnaire at 12 months were contacted to arrange an interview (563 (57%) questionnaires were subsequently completed by telephone or by interview face to face). We report here on data from the WSD telehealth questionnaire study.
Telehealth was defined within the trial as “the remote exchange of data between a patient and healthcare professional to assist in the diagnosis and management of a healthcare condition.”18 Participants used telemonitoring equipment to collect and transmit vital signs data. These data were classified into risk related alerts (for example, using a traffic light system), according to parameters that would be set initially on the basis of clinical guidelines or by a clinician responsible for the patient’s care. These parameters were reset by a clinician (general practitioner, telehealth nurse, or community matron) as required, after an initial settling-in period. The exact response to the alert depended on the risk level associated with the readings, clinical judgment, and local protocols that were usually based on clinical guidelines.18
Monitoring staff were also able to transmit health related questions, messages, or videos to educate patients on their conditions, using the telehealth base unit or set top box. Patients in the telehealth group were not charged for using the telehealth services (for instance, freephone numbers were provided for calls to central monitoring teams, or for transmitting vital signs data). But these patients were expected to have or to arrange for a telephone line, power points, and electricity; patients at one site were expected to have a television available.
The telehealth systems thus included both telemonitoring and telephone support. The trial was not designed to investigate the effect of individual service configurations or technologies.18 Rather, we sought to understand whether “telehealth,” as a class of technologies added to standard support and treatment, is cost effective compared with standard care alone. However, each study site had different suppliers and service models, which evolved over the course of the trial.
Telehealth equipment included a base unit (freestanding or a television set top box) and peripherals such as weigh scales, pulse oximeters, blood pressure cuff, and glucometers. The peripherals could communicate with the base unit either wirelessly or by cable. To set the costing methods and issues in context, the web appendix provides a brief overview of the service models used in 2009-10.
Costs of delivering telehealth interventions
We calculated the per person costs to purchasers of the telehealth equipment and support provided within the trial.
Information on each participant’s telehealth equipment was obtained from sites, along with prices paid. In two sites, a stock of equipment was purchased in advance of the trial. If equipment was purchased rather than rented, costs of base units were annuitised over five years, and costs of peripherals annuitised either over the lifetime of the item (if information was available from sites or manufacturers’ specifications) or over five years. Five years has been used as the standard lifetime for both computer technology and “short life medical and other equipment” in NHS (the UK’s health service) capital accounting.22 This same duration has been used in other telemedicine studies.23 We raised prices to 2009 levels by using the Hospital and Community Health Services’ prices inflator.24 Site 3 provided information on the monthly rental charges for four different combinations of peripherals and on the cost of pulse oximeters and glucometers purchased separately.
Telehealth support costs
Telehealth support costs included personnel involved in monitoring and responding to telehealth “triggers,” supervision of monitoring staff, back office functions, project management (planning, contract supervision, monitoring), and staff training. Cost estimates excluded posts (or parts thereof) associated purely with trial recruitment or evaluation support. From each site, we obtained the information needed to calculate overheads on directly provided staffing costs, including on-costs and indirect administrative and premises costs. If projects were unable to quantify indirect administrative overheads, these were estimated as 16% of direct salary costs.24 Costs relating to technology support (for example, for accommodating delivery personnel) were calculated by following costs methods used by the Personal Social Services Research Unit for capital overheads.24
Costs of central telehealth monitoring teams, whether directly provided or contracted out, were calculated top down, dividing total expenditure by the annual number of service users. We carried out top down costing because of the variety of interventions and rapid changes in delivery models that occurred in response to the trial timetable. By contrast, costs of contributions by “local” telemonitoring staff in two sites were calculated bottom up. In site 1, data provided by the sites was used to estimate that 15% of trial participants were monitored by community matrons and specialist nurses; and that in site 2, 24% were monitored by community matrons and specialist nurses.
Much of the work by local monitoring personnel involved direct patient contact for reasons not necessarily associated with telemonitoring. This contact, whether response related or not, might be observed and reported by questionnaire participants. To avoid double counting, only the time spent by local personnel in monitoring the telehealth screen and in training on how to use the telemonitoring system was costed. The average duration of local monitoring time per patient day (2 min) was provided by sites’ project teams, and used to calculate the total time spent per year, to which we attached relevant unit costs.
We calculated local staff costs from information provided by project teams on numbers of staff and their pay bands,25 using band midpoints. On-costs and nationally applicable overheads (capital, indirect, and direct) were added.24 Total monitoring costs from both central and local teams were combined and divided by the total number of participants monitored in the year, providing an average monitoring cost per participant per year.
The costs of installers and engineers (at sites 1 and 2) or the contract cost for installation and maintenance (at site 3) were split into fixed and variable components. We divided these costs by applying proportions calculated from detailed information provided by one site on the expenditure breakdown (2009-10) on activities related to installation, de-installation, and maintenance. This analysis suggested that 90% of costs related to installers (and their associated administrative and capital overheads, plus costs of equipment support in terms of transport and storage) are fixed and spread over five years. The remaining 10% was considered to be incurred during that year only (2009-10). Calculations also included expenditure associated with software licences, maintaining servers, and freephone telephone numbers, and arrangements for transmitting data from the telehealth base units.
We combined the average costs per person of central and local monitoring, installation, and maintenance (including administrative, premises, and capital overheads), to calculate the cost of telehealth support per intervention participant per site. This cost of telehealth “implementation” therefore varied between sites, and was added to the individually varying equipment costs. A small number of participants randomised to the control group but who had received telehealth equipment were also allocated direct intervention costs.
Self reported service use data
We applied national unit costs (2009-10 prices) from published sources to self reported data on service use (table 1⇓). Information for most unit costs for community health and social care was taken from the Personal Social Services Research Unit24; national reference costs were applied to hospital based services.26 All costs associated with self reported service use collected for the three months before 12 month follow-up were multiplied by four to give a yearly equivalent for the cost effectiveness analysis. Costs included were assumed to be incurred by health and social care agencies even if patients contributed copayments (such as for dentistry, chiropody, and optician services). For equipment and adaptations, only costs to public organisations were included, and we excluded costs reported as incurred by the patients or their families.
Service use and costs are reported as means and standard errors, unless otherwise stated. Where descriptive statistics are presented, differences are given as raw differences (£) between group means and as standardised differences (the difference between group means, divided by the standard deviation of the total sample,33 presented in percentage terms)
The primary outcome for the cost effectiveness analysis was the incremental cost per quality adjusted life year (QALY) gained, constructing utility values from the EQ-5D34 with societal weights (the York A1 tariff).35 36 We calculated QALYs by using an area under the curve analysis, with linear interpolation of utility scores between baseline and 12 month assessments.37
The secondary outcome was ICECAP-O,38 a capability index for older people that measures quality of life along five dimensions: attachment, security, role, enjoyment, and control. Attribute levels were valued for people aged 65 years and over (1=full capability, 0=no capability).
We explored two other outcomes. The short form of the Spielberger State-Trait Anxiety Inventory (Brief STAI)39 measures “state anxiety” and has been widely used, including for people with diabetes.40 The measure was rescaled in our analysis to a 0-1 range of possible levels of anxiety (0=lowest, 1=highest; original score range 6-24). The short form Center for Epidemiologic Studies Depression scale (CESD-10)41 is a screening instrument for depression symptoms. The scale ranges from 0 to 30; a difference of five points or more has been interpreted as clinically meaningful (that is, showing depressed symptoms).42 All outcome measures described were assessed at baseline, at short term follow-up (four months), and long term follow-up (12 months).
The economic evaluation adopted a health and social services perspective. Analyses were carried out in Stata version 11.43 The cost effectiveness analysis was based on the estimation of net benefit regressions.44 45 46 We constructed a model of net monetary benefit that was suitable for clustered data, to explore the probability that telehealth is a cost effective addition to standard care. This probability was calculated across a range of assumed values for decision making in health and social care, based on willingness to pay for an incremental outcome gain. The models adjusted for baseline costs, baseline utility,37 or baseline secondary outcome measure; and for site, age, sex, ethnicity, IMD score (including the proportion of trial patients who scored in each fifth of the IMD score range, (indicated as IMD groups 1-5)),19 and two indicators of health need (one constructed from a range of chronic conditions sourced from acute hospital records,47 and the index condition).18
Net monetary benefit was defined as (ΔE)×λ−ΔC (where ΔE=additional (or incremental) outcome associated with the telehealth intervention, ΔC=additional cost of telehealth, and λ=willingness to pay per unit of outcome gain). The net benefit approach allows costs and outcomes to be considered on the same monetary scale; net benefit regressions take account of sampling uncertainty and adjust for the covariates noted above.46
Using results from the net benefit regressions, we estimated incremental cost effectiveness ratios (ICERs)—that is, the additional cost per unit of outcome from the addition of telehealth to standard care. We also plotted cost effectiveness acceptability curves, depicting the likelihood that telehealth is cost effective given different assumptions about willingness to pay for outcomes.
ICERs were estimated by finding the level of willingness to pay where net monetary benefit equals zero, at which point the probability of cost effectiveness becomes 50%.45 39 The slope of the line for net monetary benefit, as a function of willingness to pay thresholds, estimated the difference in effect between groups.45 46 Telehealth should be interpreted as cost effective if the ICER is below some maximum level of willingness to pay for a unit of outcome (or if it is associated with both reduced costs and improved outcomes). In the analysis, willingness to pay values ranged from £0 to £95 000 (€110 000; $144 000) per QALY. These values included the range associated with National Institute for Health and Clinical Excellence (NICE) recommendations for using health technologies in the NHS (willingness to pay £20 000 to £30 000 per QALY).48 49
We used methods that reflected the cluster randomised nature of the trial, to avoid bias in the standard errors of regression coefficients.50 51 A multilevel approach was taken, investigating population averaged models of cost and effect. Generalised estimating equations models 52 53 were fitted using Stata command xtgee, with the general practice as cluster identifier. Models specified a log link, assuming a gamma distribution and an equal or exchangeable correlation structure, and included semi-robust standard errors. We defined incremental net monetary benefit as the difference between groups or the average marginal effect of the intervention, estimated using the “margins” command.
Missing data for costs and outcome were imputed using the multiple imputation function in SPSS version 19. Because not all participants completed every item in the Client Services Receipt Inventory, we imputed costs by categories (table 2⇓). Multiple imputation models included predictors from the outcome measures at all time points, including measures of health related quality of life (EQ-5D) and well being (ICECAP-O), psychosocial measures (measures of depression, anxiety, self efficacy), sociodemographic variables from the analysis (age, sex, site, IMD, ethnicity), and trial related variables (treatment allocation, reasons for withdrawal from the trial, and costs at baseline or at 12 month follow-up (in the categories given in table 2).
Five participants who completed the outcome measures but did not complete any questions on service use were dropped from the analyses. We did multiple imputation of missing observations by sampling from an identified subset of data having similar values to the unit with missing data, to create 10 complete datasets. These data were first analysed and then combined to produce unbiased results.54 55 The analysis adhered to the intention to treat principle, with participants grouped according to their randomised allocation, although a few patients in the intervention group received usual care and some controls received telehealth.
Sensitivity analyses—decreases in the costs of equipment
Equipment costs might have a considerable effect on the overall costs of telehealth, and conclusions about cost effectiveness could depend on the unit cost of equipment use. Equipment prices may fall over time as technology evolves. We explored the effect of falling input prices, using data we obtained from the Department of Health on equipment prices in North American markets in 2010. We applied general price decreases of 50% and 80% to equipment costs calculated for the trial. Because North American equipment prices were 10-50% of the price for equipment purchased in England before the trial, these assumptions were relatively conservative.
Sensitivity analyses—“at capacity” scenario
Telehealth teams may have been able to work at higher capacity. We initially planned to monitor about 1000 patients per site for a few months during the trial, as those in the intervention group were gradually joined by those in the control group. However, teams were monitoring about half to three fifths of this original target in 2009-10. Sensitivity analyses explored the costs of a service that monitored 1000 people in each site, on the assumption that central teams would not have had to increase staff complement to cope with additional demand, and that service structure and patient outcomes would not have changed at this larger scale (table 3⇓).
The two parameters—equipment costs and telehealth support costs—were varied as has been described and entered into the statistical analyses, using the same models and covariates as in the main analysis.
Of 3230 participants in the WSD telehealth trial, 1573 participated in the WSD telehealth questionnaire study: 845 were randomised to the telehealth intervention and 728 to usual care.20 Seventeen people who were randomised to usual care received telehealth, and six randomised to telehealth did not receive any equipment. At baseline, data for service use were available for 841 intervention participants and 728 usual care participants. At 12 month follow-up, outcomes data were available for 974 participants, of whom 969 had costs data available (538 intervention, 431 control; table 1). Costs data at baseline and 12 month follow-up were available for 965 participants (534 intervention, 431 control).
By 12 month follow-up, 599 (38%) participants had dropped out of the questionnaire study. Baseline characteristics were grouped by participants with available economic data at baseline, those who completed the study instruments at 12 months, and those who did not complete the 12 month questionnaires (table 4⇓). In terms of these characteristics, the groups were broadly similar at the outset of the trial, although there was a significantly larger proportion of people with heart failure in the usual care group than in the telehealth group (38% v 31%; z=2.6894, P=0.0072). The telehealth group also had a larger proportion of patients with chronic obstructive pulmonary disease than the usual care group (40% v 34%; z=−2.5087, P=0.012).
There were also differences between intervention and usual care participants in relation to the IMD groups 1 and 2 (that is, the two least deprived groups)19 (table 4), but mean scores did not differ substantially. Within each treatment group, the baseline and follow-up samples were broadly similar in age, number of comorbidities, health and social care costs, the proportion of women, and proportions of patients with chronic obstructive pulmonary disease and heart failure.
There were statistically significant differences between the baseline and follow-up telehealth samples in terms of mean IMD score, the proportion of patients within IMD group 5 (that is, the most deprived group), and the proportion of patients with an index condition of diabetes (table 4). The balance of long term conditions in the intervention group thus shifted somewhat over time, but at both baseline and follow-up, participants with chronic obstructive pulmonary disease made up the largest group; in the control group, participants with heart failure made up the largest group. The proportion of participants in the sample from IMD group 5 was significantly lower at follow-up than at baseline (15% v 20%; z=2.34, P<0.05; table 4). This difference between follow-up and baseline was also reflected in the mean IMD scores in the intervention group (26.1 v 27.7; P=0.046; table 4). The control group had no significant differences between characteristics at baseline and at follow-up completion.
Service use and costs
Service use was summarised under broad categories (table 1). Individual items of service use were not imputed, thus mean values (not adjusted for case mix) were presented for non-missing cases. Reported use of most services was broadly similar between the telehealth and usual care groups, although the standardised difference between groups for emergency department services exceeded 10%. There was a broad pattern of slightly fewer reported contacts with services for the telehealth group than for the usual care group.
Table 3 lists the annual telehealth intervention delivery and equipment unit costs, and intervention unit costs excluding those costs relating to project management. There was considerable variation across sites, partly because of differences in the extent of contracting out, project management structures, and lead partner. Based on these unit costs, the average annual cost per participant for telehealth equipment and support was estimated as £1847 (standard error £11.3), for participants who had received equipment and for whom costs data were available at 12 month follow-up.
Table 2 presents costs of self reported service use over the last three months before 12 month follow-up, by category; mean values summarise the costs derived from the imputation process. Excluding the direct costs of the intervention, hospital costs made up about half the total costs for all participants, followed by primary care costs (about 18%); combined costs of social care, day care, and equipment (about 16%); and drugs about (18%). Excluding intervention specific costs, costs in the telehealth group were lower than those in the usual care group, with a standardised difference in costs of about 12% between groups. If direct intervention costs were included, costs in the telehealth group were higher than in the usual care group (standardised difference of 10%). For the intervention group, the three month costs for direct equipment averaged £169 per person, and other direct costs of telehealth were £289 per person—representing 18% and 11%, respectively (table 2). Total costs for health and social care, for the last three months before the 12 month interview, were £1139 and £1380 for the telehealth and usual care groups, respectively, excluding the direct costs of the intervention; if direct costs were included, these costs were £1596 and £1390, respectively.
Table 5⇓ shows costs and outcomes data from the net benefit analyses, as well as corresponding raw mean values for base case costs and outcomes (n=965). The difference between treatment groups in raw mean QALY was small. In the adjusted net benefit model, we saw little difference between the groups in this primary outcome at 12 month follow-up (mean difference 0.012) or in ICECAP-O (0.012). On the CESD-10 and Brief STAI adjusted mean scores, the telehealth group scored slightly higher than the usual care group (0.128 and 0.042, respectively; table 5). Costs including intervention costs were higher among the telehealth group than the usual care group.
Analyses for net benefit regression showed an ICER of £92 000 per QALY (table 5). Excluding project management costs, the ratio fell to £79 000. For other outcome measures, the ICER for an improvement from no capability to full capability on the ICECAP-O scale was £98 000. The ratio for an improvement from highest to lowest levels of anxiety on the Brief STAI scale was £27 000; for the CESD-10 scale, the ICER was £9000 for achieving a five point reduction.
Whether telehealth is considered to be cost effective will depend on the willingness to pay for the outcomes generated. Figure 1⇓ presents the probability that telehealth would be seen as cost effective as an addition to usual care, using an acceptability curve for different values of willingness to pay. At the £30 000 threshold (associated with NICE recommendations41), the probability of cost effectiveness was 11%. Figure 1 also shows the probability of cost effectiveness if costs related to project management were excluded: at the £30 000 threshold, the probability of cost effectiveness was 17%. Indeed, this probability including management costs only exceeded 50% at threshold values of willingness to pay above £90 000. Excluding project management costs, the probability exceeded 50% only at values above about £79 000.
In relation to an improvement from no capability to full capability on the ICECAP-O index, telehealth would not be cost effective at values of willingness to pay below £95 000 (fig 2⇓). Although there were larger differences between intervention and control groups in state anxiety and depression symptoms, they were difficult to interpret. The probability of cost effectiveness for a 100% improvement from highest to lowest levels of anxiety on the Brief STAI only exceeded 50% at willingness to pay levels above about £27 000 (fig 3⇓). The probability that the treatment was cost effective in achieving a five point reduction on the CESD-10 scale exceeded 50% at levels of willingness to pay above about £9000, and reached 90% at about £30 000 (fig 4⇓).
Reductions in equipment costs: full utilisation
Equipment costs contributed a sizeable proportion of direct costs per person for the telehealth group (table 2). Table 2 also presents the three month costs estimated for the net benefit sensitivity analyses (multiplied by four here to give the yearly equivalent). If equipment prices fell by 80%, estimated mean costs per year (unadjusted) for the telehealth group fell from £6384 (standard error £355) to £5845 (£354). However, total costs of the telehealth group remained slightly higher than those of the usual care group (difference £299, standardised difference 4%). If equipment prices decreased by 50%, total costs for the telehealth group were also higher than for the usual care group (£496, 6%). Under the 80% reduction in equipment costs scenario, the ICER fell to £52 000 per QALY (table 5). In the scenario where the service was working “at capacity,” the annual mean costs of the telehealth group fell to £5909 (standard error £354).
Reduction in equipment costs and full utilisation (combined scenario)
The two sensitivity analyses were also combined. At an 80% reduction in equipment costs and a reduction of support costs associated with working at higher capacity, the difference between groups decreased. Total mean costs of telehealth per year (unadjusted) per participant were £166 (standardised mean difference −2%) less for telehealth (at £5370 (standard error £354)) than for usual care (three month costs are shown in table 2). At a 50% reduction in equipment costs with the same decreased labour costs, the corresponding cost was £31 less (−0.40%) for the telehealth group (at £5572 (£354)) than for the usual care group (table 2). However, in the adjusted model of costs derived from the net benefit regression analyses (table 5), the costs remained higher for the telehealth group than for the usual care group, assuming 80% and 50% reductions in input price and higher working capacity (increases of £109 and £308, respectively).
With an 80% reduction in equipment costs and operating at the higher capacity, the cost effectiveness ratio fell to £12 000 per QALY. Figure 5⇓ shows cost effectiveness acceptability curves for all sensitivity analyses. No substantial changes to the results were seen: assuming an 80% reduction in equipment costs, the probability that telehealth was cost effective was 34% at a willingness to pay level of £30 000 per QALY. Results from the sensitivity analyses based on operating at increased capacity were similar. However, combining the two scenarios increased the likelihood that telehealth was cost effective, to 61% for a willingness to pay of £30 000 per QALY.
The WSD telehealth trial was the largest pragmatic, randomised controlled trial of telehealth in England, with cost and outcome data at 12 months for 969 participants. Costs of self reported service use, combined with telehealth intervention costs, were greater for the group randomised to telehealth in addition to standard care than for the group randomised to usual care alone. In a model adjusting for demographic characteristics and level of need, this difference in costs was considerably (although not significantly) greater if project management costs were taken into consideration. For the primary outcome measure, the probability that telehealth was cost effective was relatively low, only exceeding 50% at willingness to pay values above £90 000. On the secondary outcome measures of anxiety and depression symptoms, the probability of cost effectiveness rose above 50% at willingness to pay values in excess of £27 000 and £9000, respectively.
Strengths and limitations
A limitation of self report data on service use is that respondents may have under-reported, particularly if they are frequent users of a service.56 57 However, self reporting remains the only way to collect data for a wide range of health and social care services, since administrative data are agency or service specific and not always reliable. It has been recommended that a shorter period of recall is used for frequently used services,56 and in this study, we used a three month timeframe. We assumed that costs between nine and 12 months could be multiplied up to a yearly cost. This estimation made our cost effectiveness findings conservative; longitudinal hospital data have shown that initial differences between groups in bed days narrowed over the period of the intervention.47 However, the pattern associated with acute hospital services cannot be assumed to hold with services that are more frequently used and less episodic, such as community nursing or home care.
The extent to which the costs and outcomes differed between those participants who completed the 12 month follow-up and those who did not is not known. By adjusting for demographic and cost covariates at baseline that might influence the decision to complete long term follow-up, our analysis goes some way to address any dropout imbalances between intervention and control groups.
The WSD telehealth interventions were complex,58 involving both human services and advanced assistive technologies. A number of issues were likely to arise in the economic evaluation of such complex interventions: users might be a heterogeneous group; users could be highly involved in the production of care; the more active the user involvement, the more complicated the association is between inputs and outputs; and multiple agencies could be involved in delivering the intervention.59 The intervention involved coproduction by teams that varied in composition from site to site.
Heterogeneity inevitably arose from differences in the way the interventions were delivered. There also could have been variations in the mix and balance of mainstream services within and between health and social care providers in the sites. Although use of the intervention at multiple sites improved generalisability, it was more difficult to specify the intervention to be used and identify which features might have been more helpful in improving health related quality of life. The trial was not intended nor powered to examine differences in outcomes between specific service delivery models, although this could be a secondary analysis. However, there were core features of the telehealth intervention across sites: store and forward systems, patient education protocols, computerised risk based classification of vital signs data, and central monitoring teams. Whether implementation of this “disruptive” technology60 at these sites caused any system wide change in the delivery of local health and social care services is beyond the scope of this paper. However, other research in the WSD study has examined the effects of telehealth on organisations and professionals.18
Our results focus on self reported outcomes and resource use, and do not include surrogate measures of outcome such as levels of glycated haemoglobin (HbA1c), blood pressure readings, or mortality (although mortality is examined elsewhere).40 Recent reviews and studies have identified promising results from trials of telehealth in a variety of long term conditions including diabetes, heart failure, chronic obstructive pulmonary disease, and asthma. 8 9 13 15 61 62 However, the bulk of this evidence is based on results measured by surrogate and mortality outcomes, rather than by self reported data on health related quality of life. Systematic reviews have reported mixed evidence in favour of telehealth in terms of outcomes of health related quality of life for people with diabetes8 and respiratory conditions.11 63 Evidence has also favoured telemonitoring for people with coronary heart failure,9 not least because of the diversity of generic and condition specific measures reported.
It is also important to consider the country context when comparing these results with previous studies, many of which were US based. That healthcare is free at the point of use in the UK may mean that participants had better access to appropriate primary care services than a comparable population of users in the US; thus, there is less potential to reduce the use of the more expensive services in secondary care. In this study, we noted a non-significant reduction in secondary care costs in the telehealth group. Another way in which the population might have had less room to show improvement was in terms of the level of need, or severity, of the index condition. Again, the study was not designed or powered to examine the effectiveness of telehealth within condition specific subgroups; however, quality of life instruments for specific index conditions were used, and are an area for further analysis.
One question arising from these results would be that the timeframe of the evaluation may have been too short to show improvements in health related quality of life, and is a potential weakness shared with many published economic evaluations of telemedicine.16 Similarly, there is no evidence base to show that a longer time horizon leads to improved outcomes. However, QALY gain could be modelled over an extended time horizon, reflecting the different mortality rate between trial arms identified in a concurrent study of the wider population in the parent telehealth trial.47
This study raises some questions for further research: what is the extent to which telehealth should be targeted towards specific patient populations and subpopulations, and what is the association between area level factors, patient characteristics (demographics, needs levels for each index condition), and variations in their service use and costs? We plan to examine these associations in further analyses of the trial data.
There was a more extensive data collection involving unpaid care, based on questionnaires completed by the carers of respondents, as well as non-resource costs such as transfer payments to respondents: an analysis of costs from a broader societal perspective is planned using these data.
Comparison with other studies
Information on the costs of providing telehealth in the form of telemonitoring has been scarce. Direct intervention costs of telehealth (whether by telephone support or telemonitoring) reported in the literature range widely, and come from a variety of health systems and countries. Inglis and colleagues9 identified a small number of studies of telemonitoring for heart failure that gave such details. One66 noted that the costs of telemonitoring increased the total costs for the intervention group, but did not give the actual intervention cost; another67 provided a mean annual cost per patient for telemonitoring of €185.9 Barlow and colleagues68 provided UK based estimates of telehealth equipment costs of about £700-900 and monitoring costs of £260-520 per year (2007 prices). Estimated annual costs of telehealth monitoring, support, and equipment in our study varied between sites (about £1500-2000), reflecting the heterogeneity in the models of telehealth delivery.
Because there are no societal thresholds for ICERs involving ICECAP-O, Brief STAI, or CESD-10, we can only interpret any positive findings related to these instruments with caution. ICECAP-O is a relatively new instrument and little empirical information currently exists on the average values expected in a population with long term conditions, as well as on its use in economic evaluations.69 70
Implications for clinicians and policymakers
Our results suggest that the QALY gain by people using telehealth in addition to standard support and treatment was similar to those receiving usual care, and that total costs for the telehealth group were higher than for the usual care group. The probability of cost effectiveness judged by reference to this QALY measure was relatively low over a range of values of willingness to pay. Total costs were sensitive to the costs of the intervention, reducing the point estimate of the cost per QALY substantially to about £12 000 (assuming that returns to scale could be achieved without altering outcomes). However, because the difference in total costs between treatment groups was not significant even with these assumed reductions, the probability of cost effectiveness was only about 61% at the £30 000 threshold of willingness to pay, used as a reference by NICE. These results take into account costs to both health and social care systems, to give a picture of the consequences to costs and quality of life from investment in telehealth across the agencies. If investment in telehealth falls mainly to primary and social care purchasers, while most savings accrue to the acute sector—for which there is some weak evidence here—then reinvestment into community health and social care services would be vital.
What is already known on this topic
Despite accumulating evidence on the effectiveness of telehealth, there is less evidence on the effect of telehealth on service use and costs
Few telehealth evaluations have examined the association between outcomes and costs, and the evidence base presently includes studies of poor quality design and small sample sizes
Much existing evidence is based in the United States, and its applicability to care systems in the United Kingdom is questionable
What this study adds
A community based, telehealth intervention is unlikely to be cost effective, based on health and social care costs and outcomes after 12 months and the willingness to pay threshold of £30 000 per QALY recommended by NICE
A reduced cost of telehealth per QALY may be possible by combining the effects of equipment price reductions and increased working capacity of services;
On the assumption of reduced equipment costs and increased working capacity, the probability that telehealth is cost effective would be about 61%, assuming a willingness to pay threshold of £30 000 per QALY
Cite this as: BMJ 2013;346:f1035
We thank the trial participants for their time and interest in the study; the health and social care managers and professionals in Cornwall, Kent, and Newham; the participating case study organisations for their help; Alan Glanz (Department of Health) and Chris Ham (King’s Fund) for their support throughout the study; and the reviewers for their comments and suggestions.
Whole System Demonstrator evaluation team members: Stanton P Newman (principal investigator), Martin Bardsley (The Nuffield Trust), James Barlow (Imperial College London), Jennifer Beecham (London School of Economics), Michelle Beynon (City University London/University College London), John Billings (The Nuffield Trust), Andy Bowen (University of Manchester), Pete Bower (University of Manchester), Martin Cartwright (City University London/University College London), Theopisti Chrysanthaki (Imperial College London), Jennifer Dixon (The Nuffield Trust), Helen Doll (University of East Anglia), Jose-Luis Fernandez (London School of Economics), Ray Fitzpatrick (Oxford University), Catherine Henderson (London School of Economics), Jane Hendy (University of Surrey), Shashivadan P Hirani (City University London/University College London), Martin Knapp (London School of Economics), Virginia MacNeill (Oxford University), Lorna Rixon (City University London/University College London), Anne Rogers (University of Southamptom), Caroline Sanders (University of Manchester), Luis A Silva (City University London/University College London), Adam Steventon (The Nuffield Trust).
Contributors: CH, MK, and JB contributed to the planning of economic data collection and administration. CH conducted the economic analyses under the supervision of MK and J-LF. CH, MK, and J-LF reported the analyses. HD, SPN, MK, RF, JH, PB, and AR contributed to planning the overall trial design. SPN is the principal investigator for the Whole System Demonstrator trial; HD is the guarantor of statistical quality for the trial as a whole. MK is the chief investigator for the economic evaluation. SPN, SPH, MBe, MC, and LR contributed to the planning and administration of questionnaire trial data collection. SPH, MC, MBe, LR, and AS maintained and provided data for participants. SPH, MC, MBe, LR, SPN, AS, MBa, CH, MK, and J-LF contributed to planning the analyses. All the authors reviewed the manuscript. The evaluation team met regularly during the trial period and contributed as a whole to discussions of the data under collection.
Funding: This is an independent report commissioned and funded by the Policy Research Programme in the Department of Health.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Department of Health for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: The study was approved by the Liverpool NHS research ethics committee (reference 08/H1005/4).
Data sharing: No additional data available.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.