Rapid Responses to:

INFORMATION IN PRACTICE:
Pamela S Whitten, Frances S Mair, Alan Haycox, Carl R May, Tracy L Williams, and Seth Hellmich
Systematic review of cost effectiveness studies of telemedicine interventions
BMJ 2002; 324: 1434-1437 [Abstract] [Full text]
*Rapid Responses: Submit a response to this article

Rapid Responses published:

[Read Rapid Response] Telemedicine too important to be derailed by flawed research
Albert J Kirshen   (15 June 2002)
[Read Rapid Response] Misleading title in "This week in the BMJ"
Patricia A Macnair   (16 June 2002)
[Read Rapid Response] Telemedicine field is ready for forthright analyses, including both positive and negative
Pamela Whitten   (18 June 2002)
[Read Rapid Response] Cost-utility analysis is not always appropriate
Paul McCrone   (18 June 2002)
[Read Rapid Response] Failure to compare cost effectiveness of civilian against military telemedicine is a red herring
Carl R May   (18 June 2002)
[Read Rapid Response] Re: Economic evaluation is a broad church
Alan R Haycox   (20 June 2002)
[Read Rapid Response] Telemedicine: Is cost effectiveness more important than clinical effectiveness and quality of life?
Giuseppe Riva   (20 June 2002)
[Read Rapid Response] Half of cost effectiveness is cost
Robert C Hsiung   (21 June 2002)
[Read Rapid Response] The Evidence Base Supporting The Clinical Effectiveness of Telemedicine Also Needs Strengthening
Dr Frances S Mair   (22 June 2002)
[Read Rapid Response] Wasted times for telemedicine
James Sherifi   (26 June 2002)
[Read Rapid Response] Quality of systematic reviews of cost-effectiveness studies
Paul A Scuffham   (27 June 2002)
[Read Rapid Response] Lack of evidence for effectiveness differs from evidence of lack of effectiveness
Alan R Haycox, Frances S Mair, Senior Lecturer, Department of Primary Care   (4 July 2002)

Telemedicine too important to be derailed by flawed research 15 June 2002
 Next Rapid Response Top
Albert J Kirshen,
Assistant Professor - Geriatric Medicine, University of Toronto
Temmy Latner Centre for Palliative Care, 3000-700 University Ave., Toronto ON M5G 1Z5 Canada

Send response to journal:
Re: Telemedicine too important to be derailed by flawed research

I was intrigued by the article of Whitten, Mair et al and its importance in promoting evaluation of health care technologies. The article, itself, however, has some crucial technical flaws that may, in my opinion, invalidate its conclusions.

I am particularly concerned in that not all the evaluation results sought by the authors are published in journal format, and certainly not in the locations searched. The authors do not indicate whether a concerted approach was made to governments and organizations that fund telemedicine for copies of formal evaluations. They do not indicate whether they searched the computer, business or defense literature, nor do they indicate that they searched the World Wide Web.

These lacks lead me to question the completeness of their search and, hence, the validity of their conclusions.

Misleading title in "This week in the BMJ" 16 June 2002
Previous Rapid Response Next Rapid Response Top
Patricia A Macnair,
Freelance Medical Journalist & Broadcaster
BBCi Health, Room 2470, 201 Wood Lane, London W12 7TS

Send response to journal:
Re: Misleading title in "This week in the BMJ"

Your title in This week in the BMJ which refers to this article (Systematic review of cost effectiveness studies of telemedicine interventions) is misleading. It claims that “Telemedicine is not cost effective”, so confusing lack of evidence with evidence against cost- effectiveness. As the review by Whitten et al points out, there is little published evidence to confirm whether or not telemedicine is a cost effective alternative to standard healthcare delivery.

Telemedicine field is ready for forthright analyses, including both positive and negative 18 June 2002
Previous Rapid Response Next Rapid Response Top
Pamela Whitten,
Associate Professor
Michigan State University, Department of Telecommunication, East Lansing, MI 48824-1212

Send response to journal:
Re: Telemedicine field is ready for forthright analyses, including both positive and negative

I wish to thank Professors Kirshen and Latner for their interest in our recent paper. We wholeheartedly agree that telemedicine is an important subject. As such, the authors of this paper have collectively contributed dozens of articles regarding telemedicine to peer-review outlets. Just as we have felt it important to publish those studies with positive outcomes, we feel it is equally important that we publish those pieces with less positive results. A field can only improve and advance through effective analyses.

In regard to other comments in this post, I would encourage the authors to look at the paragraph under "Search Strategies" for a comprehensive listing of the indices we searched. Our intent was to analyze articles from these more academic indices with the goal of maximizing objective articles from peer-reviewed outlets. It was not our intent to examine reports that come from such sources as government funding outlets or defense-specific literature. However, I would encourage Professors Kirshen and Latner to undertake a review of such trade or commercial publications/websites. Telemedicine is a relatively new field that will benefit from a wide range of evaluation.

Finally, I would encourage these readers to peruse the non-electronic version of this article in the BMJ for additional information.

Cost-utility analysis is not always appropriate 18 June 2002
Previous Rapid Response Next Rapid Response Top
Paul McCrone,
Senior Lecturer in Health Economics
Institute of Psychiatry, De Crespigny Park, London SE5 8AF

Send response to journal:
Re: Cost-utility analysis is not always appropriate

I read with interest the paper by Whitten and colleagues which provided a review of cost-effectiveness studies of telemedicine.(1) The paper criticised the methods used in many of the studies. The identification of definitive cost-effectiveness evidience requires robust methods and these are sadly lacking in many economic studies. However, I disagree with the implication that studies that do not adopt a cost- utility approach are flawed. Yes, in an ideal world we would have a generic measure of outcome that is equally sensitive to change for different patient groups. This though is clearly not the case. QALYs can be generated in a variety of ways, for example using multi-attribute scales such as the EuroQol or the SF-36 or by direct measurement using standard gambles or time-trade-off techniques. Valid comparisons between different treatments in different disease groups are compromised by this heterogeneity.

In addition, QALYs may not be sensitive to change for some conditions such as schizophrenia and therefore these conditions are biased against in 'QALY league tables'. In the light of these difficulties good- quality cost-effectiveness anlyses with illness specific outcome measures may well be less dangerous than the innappropriate use of cost-utility analysis.

Yours sincerely

Paul McCrone

(1) Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ 2002;324:1434-7

Failure to compare cost effectiveness of civilian against military telemedicine is a red herring 18 June 2002
Previous Rapid Response Next Rapid Response Top
Carl R May,
Professor of Medical Sociology
University of Newcastle upon Tyne, Centre for Health Services Research, 21 Claremont Place, Newcastl

Send response to journal:
Re: Failure to compare cost effectiveness of civilian against military telemedicine is a red herring

EDITOR, In his comment on our systematic review of cost-effectiveness studies of telehealthcare systems Professor Kirshen observes that we have not searched the wider government - and especially defence related - literature. My colleague Pamela Whitten offers a robust defence of our search strategy in this exercise in her response. We were concerned primarily with the peer-reviewed literature published in scientific journals. The reason for our interest will, I hope, be apparent to readers of the concluding paragraph of the article.

Professor Kirshen’s point about the value of telemedicine in defence applications is an important one, since very little is actually known about some of these systems in practice. But it's is a bit of a red herring in relation to our review’s stated objectives. There is no doubt that the armed forces of the United Kingdom, United States, France, Russia, and the People’s Republic of China amongst others, regard telemedicine systems as a cost effective solution to specific problems in the provision of health care. The British Antarctic Survey and a number of oil companies also employ telehealthcare systems for much the same reason: they may be the only possible means of providing diagnostic support and advice about treatment and management to small groups of people in remote and hostile settings.

These technologies have, in recent months, been employed to support and guide self-treatment for a life threatening disease in Antarctica, and to provide expert neurosurgical advice to a regimental medical officer in the Sierra Leonian hinterland. Doubtless British and American soldiers presently have such support on remote mountainsides in Afghanistan. (The concept isn't a new one: in various wars over the last 150 years in Afghanistan, British soldiers have used horse-mounted messengers, heliographs and short wave morse radio to do this.) Such systems allow military healthcare expertise to be globally distributed from safe central locations - but it should also be remembered that these activities are not restricted by restraints on licensure and liability experienced across national and state boundaries by civilian doctors.

The important point that needs to be made here is that battlefield and near-combat applications of telemedicine might be seen to be cost effective by armed forces not just because they distribute expertise globally and rapidly in unpredictable circumstances, but also because they do not place expert clinicians at risk of loss to enemy action. (In action, army medical officers and their assistants have a disproportionately high casualty rate.) These circumstances are structurally quite different to those in which these technologies are usually employed in civilian medicine. Civilian telehealthcare systems have been shown to be clinically effective in the rural mid-west of the USA, Australia, Canada and Norway where geography and climate intervene in the provision of care. However, the jury is still out on whether telemedicine is cost-effective because - as we show in our review - the current scientific literature is run through with methodologically inadequate studies.

Re: Economic evaluation is a broad church 20 June 2002
Previous Rapid Response Next Rapid Response Top
Alan R Haycox,
Senior Lecturer - Health Economics
University of Liverpool, L69 3GE

Send response to journal:
Re: Re: Economic evaluation is a broad church

In his response to our paper, Paul McCrone appears to criticise an aspect of our paper that I do not recognise. The thrust of our argument was that economic evidence exhibits several layers of quality. The lowest quality analyses are simple cost studies that simply provide a 'price' for a service. Next come cost minimisation studies that incorporate outcome, but simply to perhaps arbitrarily assume equivalance. The next layer of evidence concerns cost-effectiveness studies which provide a limited degree of comparison with other therapeutic areas. The broadest methodology in allowing comparisons with other therapeutic areas is cost utility analysis. Our simple message was that the vast majority of the economic evidence available was confined to the lower qualities of the evidence continuum.

With regard to our supposed implication that studies that do not adopt a cost-utility approach are flawed; I would agree entirely with Paul McCrone's analysis. Health economics is still in its infancy, and must be allowed sufficient flexibility to develop in a number of directions. The theoretical and practical criticisms underlying the use of QALYs have been frequently rehearsed in the health economics literature. I agree that "good quality cost-effectiveness analyses with illness-specific outcome measures may well be less dangerous than the inappropriate use of cost utility analysis". Unfortunately, the fact remains that, in our systematic review of cost-effectiveness studies of telehealthcare systems, both good quality cost-effectiveness analyses and cost utility analysis were conspicuous by their absence. I would have been delighted to acknowledge the value of both of these high-quality sources of economic evidence, but unfortunately this pleasure was denied me!

Telemedicine: Is cost effectiveness more important than clinical effectiveness and quality of life? 20 June 2002
Previous Rapid Response Next Rapid Response Top
Giuseppe Riva,
Head Researcher - Applied Technology for Neuro-Psychology Lab.
Istituto Auxologico Italiano, Casella Postale 1, 28900 Verbania, Italy

Send response to journal:
Re: Telemedicine: Is cost effectiveness more important than clinical effectiveness and quality of life?

I read with attention the paper by Whitten and colleagues [1], which provided a review of cost-effectiveness studies of telemedicine. However, the focus on cost effectiveness could be misleading for clinicians and institutions interested in evaluating the possibility of using telemedicine applications.

First, major new health care technologies have been introduced during recent decades without adequate cost-effectiveness assessment. Good examples of such technologies are the new imaging methods, e.g. computerised tomography and magnetic resonance imaging. Although they were soon found medically valid, their economically justifiable uses were not identified until they had been adopted extensively worldwide.

Second, as cited by the fourth point of the World Medical Association Declaration Guidelines for Medical Doctors [2], we cannot forget that the doctor's fundamental role is “to alleviate distress of his or her fellow men, and no motive, whether personal, collective or political, shall prevail against this higher purpose”. Following this line, the IOM stated in 1990 in Medicare: A Strategy for Quality Assurance [3], that "quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge" ( p. 21). The respect of this position justifies the methods used in many of the articles and their goals: verify the clinical effectiveness and the improvements in patients’ quality of life allowed by telemedicine [4].

This is why I strongly disagree with the implication that telemedicine studies that do not present data on both costs and benefits are methodologically flawed. The risk is to confound quality problems from those arising from resource availability. A key measure of "success" in health care is quality patient care and this reflects a primary reason for ICT investments. On the contrary, telemedicine studies focused mainly on cost- savings may be ethically flawed: a definition of quality of care on a sliding scale in which judgments about quality vary according to what can or cannot be afforded is ethically unacceptable.

For several applications, including teleradiology, a better quality of life and sometimes clinical benefit is obtained through avoidance of travel and associated delays. In general, telemedicine seems to offer clinical benefits in terms of increased compliance with medication and self-monitoring [5]. In a systematic review of telemedicine assessments based on searches of electronic databases between 1966 and December 2000 Haley and Colleagues [6] identified different scientifically credible studies that included comparison with a non-telemedicine alternative. More than half (56%) of the studies suggest that telemedicine has advantages over the alternative approach. In particular the most convincing evidence on the efficacy and effectiveness of telemedicine is given by some of the studies on teleradiology (especially neurosurgical applications), telemental health, transmission of echocardiographic images, teledermatology and home telecare.

However, I agree with Whitten and colleagues that most of the available literature has methodological limitations and refers mainly to pilot projects and to short-term outcomes. My feelings are that it is still true what Whitten and colleagues commented two years ago after their analysis [7] on a slightly reduced number of articles (551 in [7], 612 in [1]): “it is premature for any statements to be made, either positive or negative, regarding the cost-effectiveness of telemedicine in general.” (p. s4).

(1) Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ 2002;324:1434-7

(2) World Medical Association. Declaration Guidelines for Medical Doctors Concerning Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment in Relation to Detention and Imprisonment, 29th World Medical Assembly Tokyo, Japan, October 1975. Online (accessed 18 June 02): http://www.wma.net/e/policy/17-f_e.html

(3) IOM. Medicare: A Strategy for Quality Assurance. K.N. Lohr, ed. Washington, D.C.: National Academy Press, 1990.

(4) Hebert, M., Telehealth success: evaluation framework development. Medinfo, 2001. 10(Pt 2): p. 1145-9.

(5) de Lusignan, S., et al., Compliance and effectiveness of 1 year's home telemonitoring. The report of a pilot study of patients with chronic heart failure. Eur J Heart Fail, 2001. 3(6): p. 723-30.

(6) Hailey, D., R. Roine, and A. Ohinmaa, Systematic review of evidence for the benefits of telemedicine. J Telemed Telecare, 2002. 8 Suppl 1: p. 1-30.

(7) Whitten, P., C. Kingsley, and J. Grigsby, Results of a meta-analysis of cost-benefit research: is this a question worth asking? J Telemed Telecare, 2000. 6 Suppl 1: p. S4-6.

Half of cost effectiveness is cost 21 June 2002
Previous Rapid Response Next Rapid Response Top
Robert C Hsiung,
Associate Professor of Clinical Psychiatry
University of Chicago, 5737 S. University Ave., Chicago, IL, 60637-1507, USA

Send response to journal:
Re: Half of cost effectiveness is cost

Editor--

First, I would like to thank Whitten et al. [1] for their excellent paper and the other correspondents for their excellent discussion. I agree that the methodology used to assess cost effectiveness should be as robust as possible. Apart from issues of methodology, however, I would like to point out that half of cost effectiveness is cost and that as telemedicine adapts itself to the Internet, costs will significantly decrease.

At the recent annual meeting of the American Telemedicine Association, a number of researchers, including Whitten herself and other colleagues [2], reported that some patients are not only quite satisfied with POTS (plain old telephone system) telepsychiatry services, but actually prefer them to being seen in person. Appel et al. [3] had subjects who preferred the phone to videoconferencing. And even text-only psychotherapy is possible [4]. As more and more patients connect to clinicians from their homes, costs cannot but decrease and satisfaction may in some cases increase. Effectiveness is, of course, another matter, which brings us back to the need for methodologically sound research.

Robert C Hsiung, MD
Associate Professor of Clinical Psychiatry
University of Chicago
5737 S. University Ave.
Chicago, IL
60637-1507
USA
dr-bob{at}uchicago.edu

1. Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ 2002 June 15; 324: 1434-1437. [Full text]

2. Whitten P, Ziazi Z, Marion L. Access and outcomes through the Michigan Telepsychiatry Project. Paper presented at the American Telemedicine Association Annual Meeting 2002 June 4.

3. Appel PR, Bleiberg J, Noiseux J. Behavioral telehealth for pain management: Efficacy and consumer satisfaction. Paper presented at the American Telemedicine Association Annual Meeting 2002 June 4.

4. Stofle GS. (in press). Chat room therapy. In: Hsiung RC, (ed.). E-therapy: Case studies, guiding principles, and the clinical potential of the Internet. New York: W.W. Norton & Company.

The Evidence Base Supporting The Clinical Effectiveness of Telemedicine Also Needs Strengthening 22 June 2002
Previous Rapid Response Next Rapid Response Top
Dr Frances S Mair,
Senior Lecturer (Clinical)
Dept of Primary Care, University of Liverpool, The Whelan Building, Quadrangle, Brownlow Hill, L69

Send response to journal:
Re: The Evidence Base Supporting The Clinical Effectiveness of Telemedicine Also Needs Strengthening

Editor, first, I wish to thank Patricia Macnair for her interest in our paper and concur with her comments. Our study points out "that there is presently no persuasive evidence about whether telemedicine represents a cost effective means of delivering health care" rather than stating that "telemedicine is not cost effective".

Secondly, I would strongly agree with Giuseppe Riva that clinical effectiveness and quality of life are crucial considerations when considering the value of any new way of delivering health care. However, I believe that he has misunderstood our paper because it focuses purely on the subject of cost effectiveness and makes no attempt to suggest that this is a more important issue than clinical effectiveness or quality of life. I strongly support further investigation of the clinical effectiveness of telemedicine because, at present, there remains little firm evidence of widespread clinical benefits of telemedicine as demonstrated by recent reviews of this subject.1,2. I disagree with the suggestion that valuing clinical effectiveness and quality of life justifies the methods used by the studies included in our review. First of all, the studies cited, has as their main focus examination of cost effectiveness and secondly, many of the methodological limitations identified will also have negatively impacted their ability to inform the debate regarding the efficacy of telemedicine. 1. Currell R, Urquhart C, Wainwright P, Lewis R. Telemedicine versus face to face patient care: effects on professional practice and health care outcomes (Cochrane review). In: The Cochrane Library, Issue 3, 2000. Oxford: Update Software. 2. Hersh WR, Helfand M, Wallace J, Kraemer D, Patterson P, Shapiro S, Greenlick M. Clinical outcomes resulting from telemedicine interventions: a systematic review. BMC Medical Informatics and Decision Making 1 (5), (www.biomedcentral.com/1472-6947/1/5)2001.

Wasted times for telemedicine 26 June 2002
Previous Rapid Response Next Rapid Response Top
James Sherifi,
retired gp

Send response to journal:
Re: Wasted times for telemedicine

Dear Sir,

Whilst welcoming and generally endorsing the findings of the systematic review carried out by Pamela Whitton et al (1) on the cost effectiveness of telemedicine interventions, I do take issue with the conclusion drawn and in particular with the negative impact it may have on creating services in this area. The benefits of telemedicine consults for certain conditions outnumber the disadvantages in more ways than those based purely on cost. Each may be difficult to evaluate on its own but the sum can represent considerable savings to the NHS in finance and resources.

Some years ago in Colchester, we undertook a GP led dermatology telemedicine study that was not published (one that was not included even in the 96% that failed to meet “quality criteria”). However we did find statistically relevant changes in a number of areas including: Reduction in waiting time for specialist opinion; Reduction in GP and patient anxiety levels; High satisfaction rating by GP’s and patients; No change in morbidly or mortality related to the condition on long term follow-up; Excellence as a training tool in dermatology for GP Principals and Registrar alike with likely decrease in future referrals for specialist opinion. The empirical cost calculated per consult was approximately 50% that of a traditional out patient appointment.

Of the 55 articles with cost data reviewed in this article, 72% leant towards their being a cost benefit. This should provide the springboard for future action, led by the colleges, aimed at implementing large sale pilot studies of appropriate research quality.

Years have passed since our own study. There is still a shortage of specialists in many areas particularly dermatology. Why do we not follow the lead of primary care in installing initiatives such as nurse triage, nurse prescribing, telephone consultations, NHS Direct, walk in surgeries etc and start a service for valid reasons, looking at the cost analysis later? At times our demand for scientific rigour in research impedes progress in medical practice.

JAMES SHERIFI MBChB
Retired GP Principal
Colchester, Essex

(1)Whitton, Mair, Haycox, May, Williams, Hellminch Systemic review of cost effectiveness studies of telemedicine interventions. BMJ 2002;324:1434-1437

I have no conflicting interests.

Quality of systematic reviews of cost-effectiveness studies 27 June 2002
Previous Rapid Response Next Rapid Response Top
Paul A Scuffham,
Senior Research Fellow
York Health Economics Consortium, University of York

Send response to journal:
Re: Quality of systematic reviews of cost-effectiveness studies

Dear Editor,

The paper by Witten and colleagues [1] reviewing the cost effectiveness of telemedicine interventions was interesting and somewhat provocative.

I am concerned that the authors confuse the quality of an evaluation with levels of evidence and evaluation design (see the rapid response by Alan Haycox to Paul McCrone). Quality should not be judged on levels of evidence or if consequences were measured, but on how well these were undertaken and reported. From their review, economic evaluations of telemedicine tend to concentrate on costs and cost-minimisation ignoring benefits. This is the conservative approach, that if benefits were measured and valued, the cost-effectiveness of telemedicine would only appear better.

In addition, Witten and colleagues have rated studies based on the 'presence of a clear hypothesis'. Although this is essential for assessing efficacy, it is substantially less important for economic evaluations. Hypotheses require the null to be accepted or rejected through statistical tests; economic evaluations typically use point estimates for costs and, without variance, cannot be subjected to statistical testing. Hence, there is normally a battery of sensitivity analyses undertaken around point estimates. What is required is a clear statement of the aims, and as noted by Witten et al., most economic evaluations do clearly state the aims.

Cost-effectiveness is about the additional cost for an additional unit of benefit; it is a value judgement about whether it is worthwhile to spend the additional costs to obtain the additional benefit (i.e. the incremental cost-effectiveness ratio or ICER). A systematic review judging the cost-effectiveness of interventions is not meaningful unless we know what the ICER or incremental cost threshold is. At best, cost- neutrality, as used in cost analyses and cost-minimisation analyses, suggests that the outcomes from telemedicine and conventional healthcare services can provide be obtained from telemedicine without additional costs. It is a pity that these authors did not report their cost- effectiveness threshold before claiming "Telemedicine is not cost- effective" [2].

There were some easy questions not addressed in the review by Witten and colleagues. I am disappointed that the review did not report circumstances where telemedicine was and wasn't cost-effective. For example, 'live consultations' are resource and cost intensive compared with asynchronous consultations, but in some cases, such as the Highlands and Islands Teledentistry project [3], might be cost-effective. Likewise, telemedicine might be cost-effective for some disciplines such as dermatology and radiology, and not others.

In their critique of the generalisability of results, the issue of equity through access to healthcare services is overlooked. There are many remote communities in the United Kingdom where there are substantial barriers to access of usual healthcare services. For example, the rate of referrals for restorative dental consultations from the Shetland Islands (this requires a trip to Aberdeen) is one-tenth of the Orkney Islands rate (where a consultant visits for one to two days per annum) [3]. Teledentistry and other telemedicine interventions, even if there are additional costs, will help address the inequities that exist in access to healthcare services.

1. Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of telemedicine interventions. BMJ 2002; 324(7351):1434-7.

2. Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Telemedicine is not cost effective. In "This week in the BMJ" BMJ 2002, June 15; 324(7351).

3. Scuffham P, Steed M. An economic evaluation of the Highlands and Islands teledentistry project J Telemed Telecare 2002; 8(3):165-77.

Lack of evidence for effectiveness differs from evidence of lack of effectiveness 4 July 2002
Previous Rapid Response  Top
Alan R Haycox,
Senior Lecturer - Health Economics
University of Liverpool, L69 3GE,
Frances S Mair, Senior Lecturer, Department of Primary Care

Send response to journal:
Re: Lack of evidence for effectiveness differs from evidence of lack of effectiveness

We very much appreciate Scufflam’s thoughtful contribution to the debate generated by our paper. He is, of course, correct to emphasize the potential value of telemedicine in improving access to specialist healthcare services in isolated locations. The use of telemedicine in such locations provides one potentially valuable method of enhancing accessibility to such services. However, such issues were beyond the scope of our review. In undertaking our review we did not attempt to introduce new evidence on the cost-effectiveness of telemedicine but simply to act as a mirror reflecting the quality and quantity of evidence that was already in existence. From a personal perspective I have little doubt that in certain locations and for certain specialties telemedicine may be able to provide a cost effective method of improving both the cost- effectiveness and accessibility of health care provision within the NHS. Unfortunately, this is a personal view and it not reflected in the weight of evidence currently available in the literature. In interpreting the results of our review it is crucial to acknowledge that lack of evidence for the cost-effectiveness of telemedicine is not equivalent to evidence of lack of cost-effectiveness of telemedicine. Our review simply identified the former without seeking to make any contribution towards the debate concerning the latter.

We would, however, take slight exception to one of the points raised by Scufflam. The use of cost minimisation analysis is only a ‘conservative approach’ if the benefits arising from telemedicine consultations are at least as good and preferably better than traditional (i.e. face to face) methods of consultation. Our argument with such an approach is that such superiority cannot simply be assumed but must be based on a high quality evidence base.

One of the major weaknesses of the evidence base was its inability to identify circumstances where telemedicine was and wasn’t cost-effective. Perhaps more than in any other area of new technology the cost- effectiveness of any telemedicine service is very much location specific. This is not to say that results obtained in one location (the Highlands of Scotland) cannot be generalised to other locations (the centre of Liverpool) but that such generalisation must be undertaken with the utmost care. Specifically an impact model should be developed to distinguish between elements that contribute to the success of a telemedicine service that are generalisable (e.g. technological factors) from those that are location specific (e.g. geographical and environmental factors). All too often the success of a telemedicine service evaluated in inaccessible locations were automatically assumed to be generalisable elsewhere without testing this assumption. Our review presents a challenge to the telemedicine research community to strengthen their analyses by identifying the extent to which results obtained in one research location are likely to be replicable elsewhere.

Our review presents an early overview of the quality of research currently available in the field of telemedicine. The review is timely given the growing interest in this branch of medicine. The review presents a challenge to the telemedicine research community to strengthen their analyses. Should they do this we would welcome the opportunity to revisit our review in the future hopefully to reflect the improved quality of evidence generated in response to this challenge. The authors of the review are, in general, persuaded of the potential value of telemedicine. Our role in undertaking the review, however, was not to be ‘for’ or ‘against’ the telemedicine but simply to reflect the realities of the current evidence base available. It is our hope that in doing so we have indicated some of the areas in which the quality of this evidence base can be improved.