Papers

Cost effectiveness analysis of screening for sight threatening diabetic eye disease

BMJ 2000; 320 doi: https://doi.org/10.1136/bmj.320.7250.1627 (Published 17 June 2000) Cite this as: BMJ 2000;320:1627
  1. Marilyn James, head of health economics (research and development) unit (m.james{at}keele.ac.uk)a,
  2. David A Turner, research fellowa,
  3. Deborah M Broadbent, clinical assistantb,
  4. Jiten Vora, consultant diabetologistc,
  5. Simon P Harding, consultant ophthalmologistb
  1. a Centre for Health Planning and Management, University of Keele, Keele, Staffordshire ST5 5BG
  2. b St Paul's Eye Unit, Royal Liverpool University Hospitals, Liverpool L7 8XP,
  3. c Department of Diabetes and Endocrinology, Royal Liverpool University Hospitals
  1. Correspondence to: M James
  • Accepted 13 March 2000

Abstract

Objective: To measure the cost effectiveness of systematic photographic screening for sight threatening diabetic eye disease compared with existing practice.

Design: Cost effectiveness analysis

Setting: Liverpool.

Subjects: A target population of 5000 diabetic patients invited for screening.

Main outcome measures: Cost effectiveness (cost per true positive) of systematic and opportunistic programmes; incremental cost effectiveness of replacing opportunistic with systematic screening.

Results: Baseline prevalence of sight threatening eye disease was 14.1%. The cost effectiveness of the systematic programme was £209 (sensitivity 89%, specificity 86%, compliance 80%, annual cost £104 996) and of the opportunistic programme was £289 (combined sensitivity 63%, specificity 92%, compliance 78%, annual cost £99 981). The incremental cost effectiveness of completely replacing the opportunistic programme was £32. Absolute values of cost effectiveness were highly sensitive to varying prevalence, sensitivity and specificity, compliance, and programme size.

Conclusion: Replacing existing programmes with systematic screening for diabetic eye disease is justified.

Introduction

Various methods of screening for diabetic eye disease have been tested in recent years,112 but few studies have produced meaningful cost effectiveness data. A three centre study commissioned by the Department of Health reported relatively high costs per case of diabetic eye disease detected.1 2 Similar data are also available from the United States.5 The Department of Health study was undermined by suboptimal screening methods, and both studies disregarded the effect of pre-existing opportunistic screening on cost effectiveness. Foulds et al studied the potential savings of systematic screening, but the cost effectiveness data are difficult to verify in the absence of sensitivity data.13 Mathematical modelling has also been used to study the potential economic benefits of screening.4 6 14

Nationally coordinated screening of diabetic patients for sight threatening eye disease is being considered as part of the national service framework on diabetes, which is due to be published in spring 2001, and an economic evaluation of a programme with high sensitivity and specificity and known prevalence and compliance is therefore urgently needed.15 The Liverpool diabetic eye study was established in 1991 to investigate the efficacy of primary care based photographic screening for sight threatening eye disease and to set up a systematic service replacing the existing opportunistic programme. We present a detailed cost effectiveness analysis of the systematic and opportunistic programmes and the effect of varying disease prevalence, compliance, and sensitivity and specificity to allow generalisation of our results throughout the NHS.

Methods

The systematic screening programme uses a mobile screening unit that visits inner city general practices together with a dedicated hospital assessment clinic.16 Screening comprises three-field, non-stereoscopic photography using mydriasis; 35 mm transparencies; and validated grading. The pre-existing opportunistic service used direct ophthalmoscopy and was performed by general practitioners, optometrists, and diabetologists. There was no systematic training, central coordination, or audit, and patients with positive results were assessed in general hospital eye service clinics.

The outcome measure was the detection of sight threatening eye disease, defined as any of the following: moderate preproliferative retinopathy or worse; circinate exudates within the macula; any exudate within 1 disc diameter of the foveola; other diabetes related disease such as vascular occlusion.

Source data

Data for this analysis were taken from two studies within the Liverpool diabetic eye study. The first was a cross sectional observational study of 320 diabetic patients registered with four general practices who were examined by a consultant ophthalmologist specialising in medical retinal diseases using slit-lamp biomicroscopy (an accepted reference standard for determining need for treatment).8 16 The second study comprised an analysis of the implementation of systematic screening in Liverpool and included a structured, closed response questionnaire administered by trained observers to the first 1363 diabetic patients recruited.17 These two studies provided all data except the specificity of the opportunistic programme, which was calculated from a previous study.1 We adopted a health service perspective for measurement of costs and benefits.

Four key data variables are necessary to determine overall effectiveness in any screening programme: disease prevalence; compliance; sensitivity and specificity of the screening method; and cost. The disease prevalence applied across both the systematic programme and the opportunistic programme, but the other variables were analysed separately for the two programmes. The overall baseline prevalence of sight threatening eye disease in Liverpool was calculated as 14.1% from the cross sectional study.

Compliance with systematic screening during data collection (1995–6) was 80%. The Liverpool study has a current contracted activity of 4000 screen events a year. To achieve this with a compliance of 80%, 5000 screening invitations need to be sent. The cross sectional study showed that the systematic programme had a sensitivity of 89% (95% confidence interval 80% to 98%) and specificity of 86% (82% to 90%).16

Compliance for opportunistic screening was calculated from the questionnaire; 78% (1059/1363) reported screening in the 12 months before attending for systematic screening.16 To accurately compare systematic with opportunistic screening the costs and detection rates were also based on 5000 invitations, but a 78% compliance produces only 3900 screen events. The sensitivity of opportunistic screening was calculated from the cross sectional study and the questionnaire data.8 Seven per cent (22/320) of patients were already under the care of an ophthalmologist for eye disease detected opportunistically. This number was divided by the prevalence (14.1%) to give 49% of patients, and a sensitivity of 63% was derived by dividing by the proportion who reported opportunistic screening in the previous year (49/78).

The specificities for the various health professionals performing opportunistic screening were taken from Buxton et al (general practitioners 89%, diabetologists 96%, and optometrists 94%).1 The proportions of patients screened by each class of health professional (obtained from the study questionnaire) were used to derive an overall specificity for the opportunistic programme of 92%.

Costs

We used an ingredient approach because the costs in screening programmes are largely fixed or semifixed; recording individual patient based costings is not helpful in this situation. Capital was given a seven year life and discounted at the test discount rate of 6%. Overhead costs for hospital based activities—grading, administration, and follow up—were set at 10% (Royal Liverpool University Hospitals Trust finance data).

Costs of systematic screening were calculated on actual resource use at 1996–7 prices for 4000 screen events with additional administrative costs to call non-attendees. Costs of opportunistic screening were calculated for an activity of 3900; with no call-recall system, there are no additional administrative costs. The proportions of patients screened by their general practitioner, diabetologist, or optometrist were identified from the study questionnaire.17 Costs of the general practitioner and diabetologist components were calculated by averaging estimates of time spent on direct ophthalmoscopy by six practitioners who regularly did screening. General practitioners' costs per minute including overheads (5 minutes at £1.72/min) were taken from Netten and Dennett,18 with five minutes, additional nursing time per consultation for instilling drops and measuring visual acuity. Diabetologist costs were calculated as a percentage of the cost of a whole outpatient visit for the hospital (standard outpatient cost for 1996-7=£55). The optometrist cost was taken as the minimum sight test fee of £13.50, representing the full cost to the NHS including staff consumables and overheads. The cost of a standard outpatient appointment was also used to cost follow up assessment of patients who had positive results.

Table 1.

Outcomes of offering screening to 5000 diabetic patients in systematic programme and of screening equivalent cohort opportunistically

View this table:
Table 2.

Annual costs of each stage of systematic photographic screening for a cohort of 5000 invited diabetic patients, 4000 of whom attended

View this table:

Cost effectiveness was calculated as total cost divided by the number of cases detected and incremental cost effectiveness as the extra cost needed to generate each additional true positive result after replacing opportunistic screening by systematic screening. To test the robustness of the study we conducted a sensitivity analysis to determine the effect on cost effectiveness of varying the key variables.

Results

A baseline prevalence of 14.1% and a cohort of 5000 patients yield an assumed 705 (14.1/100×5000) true cases of sight threatening eye disease in the target population. table 1 shows the number of true and false positive and negative results calculated for each programme. table 2 shows the costs for the components of systematic screening, and table 3 presents costs for opportunistic screening based on the percentage of the sample seen by each type of screener. Total costs were £104 996 for systematic screening and £99 981 for opportunistic screening. The cost effectiveness was £209 and £289 respectively, and incremental cost effectiveness was £32 (table 4).

Table 3.

Costs of screening tests extrapolated for opportunistic screening programme for cohort of 3900 diabetic patients from a target population of 5000

View this table:
Table 4.

Cost effectiveness of opportunistic and systematic screening programmes

View this table:

Sensitivity analysis

Figure 1 shows the effect of varying the prevalence of sight threatening eye disease on cost effectiveness. If the prevalence falls the cost effectiveness of both programmes falls. At all prevalences the opportunistic programme is less expensive, but the systematic programme is more cost effective than the opportunistic programme.

Fig 1.
Fig 1.

Effect of changing prevalence of sight threatening diabetic eye disease on cost effectiveness of systematic and opportunistic screening programmes

Fig 2.
Fig 2.

Effect of varying sensitivity of opportunistic screening on cost effectiveness. Cost effectiveness of systematic screening is shown as horizontal line

A two way analysis of the effect on the systematic screening programme of varying sensitivity and specificity within previously reported 95% confidence limits 16 gave a value of £237 for low sensitivity and low specificity (80%, 82%) and £186 for high sensitivity and specificity (98%, 90%). Systematic screening is more cost effective than opportunistic screening within the 95% confidence range. Figure 2 shows the effect of varying the sensitivity of opportunistic screening on its cost effectiveness. Cost effectiveness ranged from £350 to £202. Opportunistic screening is less cost effective than systematic screening at all levels of its sensitivity up to 95%.

As compliance with systematic screening rises cost effectiveness improves, varying from £487 for 30% compliance to £176 for 100% compliance. At 54% compliance the cost effectiveness of systematic screening equals that of opportunistic screening at £289.

Increasing activity to 6000 screens a year raises the total cost for systematic screening to £139 856. The cost per screen event falls from £26 to £23, and cost effectiveness improves to £186. There is a saving of £43 per true positive case detected when screening systematically rather than opportunistically. Increasing activity to 6000 screens a year raises the total cost for opportunistic screening to £149 972 with no improvement in cost effectiveness. This makes opportunistic screening more expensive than a systematic programme.

Discussion

We directly compared the costs of pre-existing opportunistic screening with a newly introduced systematic programme. The systematic programme is slightly more expensive than the opportunistic programme but yields 157 extra cases at only £32 per case.

In an earlier cost effectiveness assessment, Buxton et al studied 3318 screen events in three UK centres using two methods: direct ophthalmoscopy by optometrists, physicians, and general practitioners and single field non-mydriatic polaroid photography.1 2 If their figures are adjusted to 1996–7 prices the cost effectiveness of direct ophthalmoscopy by optometrists is £1057, hospital physicians £1392, and general practitioners £853-£1454; the costs of hospital and community photographic screening ranged from £670 to £2084. Their disappointing results were largely due to suboptimal screening methods and a low prevalence (5.8%).2

Lairson et al studied the cost effectiveness of four screening methods in the United States.5 They also found a large difference between a photographic protocol similar to ours ($295 (£184)) and direct ophthalmoscopy by a technician ($794), with direct ophthalmoscopy over 2.5 times more expensive than photography.

Applicability

Our results can be generalised to other British photographic screening programmes. The baseline prevalence is likely to be similar throughout the country,19 as is the effectiveness of opportunistic screening. However, accurate data on sensitivity, specificity, and compliance are required to complete an analysis based on our model. Such an analysis would be valuable when applied to other current techniques including dual modality screening,20 22 optometry based programmes,23 digital photography,24 25 and automated neural net systems.26

Several factors may have influenced our results. Disease prevalence is an important determinant of cost effectiveness, but the systematic screening was always more cost effective than opportunistic screening at all values of sensitivity and specificity within the 95% confidence limits for our data (fig 1).16 Lairson et al reported similar findings.5 This is important as prevalence in the screened population will fall with each year of screening: each year the true positive group will comprise patients who develop sight threatening eye disease in that year and a number of people with false negative results from the previous year. After several years the prevalence should approach the underlying incidence of new cases a year.

Cost effectiveness is further influenced by the effectiveness of the screening tool. Varying sensitivity and specificity between the upper and lower 95% confidence limits in the systematic programme produced only a small variation in cost effectiveness. However, in the opportunistic programme the low sensitivity for direct ophthalmoscopy gave a poor cost effectiveness. Studies of the sensitivity of direct ophthalmoscopy have all reported low rates with general practitioners and untrained physicians,1 911 27 even after intensive training.28 Unacceptably high serious error rates have also been reported with a trained retina specialist.12 The best sensitivity reported to date is 65% with a trained ophthalmologist,16 and at this level systematic screening is more cost effective.

Compliance with screening greatly affects cost effectiveness, with higher rates of compliance increasing cost effectiveness. A compliance rate of 80% was achieved in the second year of the systematic programme, and this may improve with better targeting and education and the implementation of a district diabetes register. However, full coverage is probably impossible because of factors such as death, housebound patients, and high population turnover in an inner city.

Our analysis is based on an annual 5000 invitations yielding 4000 screen events. Increasing the annual activity to full coverage at 6000 screen events increases the cost of systematic screening but confers an 11% (£23) improvement in cost effectiveness because capital costs do not change. The incremental cost effectiveness becomes negative, indicating a real cost saving.

Outcome measures

We have used the number of detected cases as our measure of effectiveness. The use of this proxy measure depends on the inference that correctly and appropriately identified cases can be treated and blindness prevented. Although useful, this kind of measure does not necessarily show the full effectiveness of a programme as it reflects process rather than final outcome. Further work is required to measure cost effectiveness against long term end points such as numbers of patients treated, years of sight saved, quality of life, or numbers of blind registrations.5

In conclusion, our cost comparison implies that a purchaser can recover the costs of opportunistic screening by diverting them to systematic screening. In our opinion reallocation of resources is feasible, and purchasers can justify the small increase in costs entailed by introducing systematic screening.

What is already known on this topic

Screening for diabetic eye disease can prevent loss of sight

Screening in Britain is currently opportunistic

The cost effectiveness of systematic screening has not been properly evaluated

What this study adds

Cost effectiveness of systematic screening in primary care using a multiple 45 field photographic protocol was £209 compared with £289 for an existing opportunistic programme

The incremental cost effectiveness of replacing opportunistic screening with systematic screening was £32

Systematic screening remained more cost effective than opportunistic screening for all values of disease prevalence

Acknowledgments

We thank Mr P Kingham of Royal Liverpool University Hospitals Trust finance department for help with costs.

Contributors: SPH, DMB, MJ, and JV conceived the study and wrote the protocol. DAT and MJ performed cost modelling and sensitivity analysis. DMB performed prevalence and compliance analysis and contributed to cost modelling. SPH wrote the manuscript with contributions from MJ, DAT, DMB, and JV. MJ is the guarantor of the study.

Footnotes

  • Funding This study was funded by North West Regional grant DIF1.

  • Competing interests None declared.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
View Abstract