Using decision analysis to compare policies for antenatal screening for Down's syndromeBMJ 1995; 311 doi: https://doi.org/10.1136/bmj.311.7001.351 (Published 05 August 1995) Cite this as: BMJ 1995;311:351
- J Fletcher, senior registrar in public health medicinea,
- N R Hicks, consultant public health physiciana,
- J D S Kay, consultant chemical pathologistb,
- P A Boyd, clinical coordinator for prenatal diagnosisc
- aDepartment of Public Health and Health Policy, Oxfordshire Health Authority, Oxford OX3 7LG
- bDepartment of Clinical Biochemistry, Oxford Radcliffe Hospital,Oxford OX3 9DU
- cMaternity Department, Oxford Radcliffe Hospital, Oxford OX3 9DU
- Correspondence to: Dr Hicks.
- Accepted 5 August 1995
Objective : To compare different screening policies for Down's syndrome across a broad range of outcomes, using decision analysis, with particular reference to the role of maternal serum testing.
Design : A decision tree was used to combine data from local sources and the medical literature to predict the likely frequency of several outcomes. Sensitivity analyses were used to test the robustness of the conclusions drawn.
Setting : Oxfordshire Health Authority.
Main outcome measures : Live births with and without Down's syndrome; miscarriages with Down's syndrome; cases of Down's syndrome detected antenatally; amniocenteses performed (and associated miscarriages); direct NHS screening costs; number of women offered screening.
Results : Screening policies for Down's syndrome that include serum testing can produce better population outcomes than programmes that do not. Each option for screening for Down's syndrome that we considered had significant drawbacks. In Oxfordshire, offering serum testing to women of all ages would prevent the birth of approximately one more baby with Down's syndrome per year than would a policy of screening for women aged 30 years or more. The cost of preventing this one extra Down's birth would be one or two normal babies lost after amniocentesis, 4500 blood tests for young women (with the associated anxiety and counselling), approximately 200 false positive serum test results and amniocenteses (with the associated anxiety and distress), and £90000 for the extra tests, counselling, and amniocenteses. Opinions are divided as to which policy is the better option for the population.
Conclusions : Decision analysis is a useful tool for determining the likely consequences of different policy options across a broad range of outcomes. This focuses debate and decision making on outcomes of care, which in turn makes it clear that the choice of screening programme for Down's syndrome depends on the relative importance ascribed to the different outcomes. If individuals' values vary widely it may be impossible to find one screening policy that meets the needs of all pregnant women.
Decision analysis is a useful method for determining the likely consequences of different policy options across a wide range of outcomes
Programmes that include serum testing can produce better population outcomes than pro- grammes that do not
Whether the benefits of having a serum test for Down's syndrome outweigh the risks for an individual woman depends on her perceptions ofthe relative importance of the different possible outcomes
If women's values vary widely it may not be possible to find one policy that meets the needs of all pregnant women
Maternal serum concentrations of various analytes including (alpha) fetoprotein, oestriol, and human chorionic gonadotrophin can be used to estimate the probability of a fetus having Down's syndrome. Wald and colleagues have predicted and subsequently shown that information derived from measurements of various combinations of such analytes, when interpreted in the knowledge of a woman's age and the gestational age of the fetus, allows a more accurate estimation of the risk of a fetus being affected with Down's syndrome than does risk estimation based on maternal age alone.1 2 This has raised the possibility of introducing biochemical testing as a screening test for Down's syndrome for some or all pregnant women. However, there is no consensus among health authorities in Britain as to whether biochemical screening for Down's syndrome should be offered and if so to which groups of pregnant women.3 The main issues that have been the topics of professional and public debate are the ethics of prenatal screening; the performance of biochemical screening tests; the choice of test; the relative costs, both personal and monetary; whether centres which introduced biochemical screening early have done the right thing; and the importance of counselling.4 5 6 7
The consequences of screening for Down's syndrome are various. They may include changes in the number of Down's syndrome babies detected, the number of Down's syndrome babies born, the number of unaffected babies born, the number of pregnancies lost by miscarriage, the amount of anxiety generated, and the direct and indirect financial costs of the programme to the NHS, other agencies, and pregnant women and their families. Most published contributions to the debate have provided information about only one or two measures of outcome such as detection rate1 or psychological costs.5 8 Others have used summary measures such as overall cost or saving per Down's syndrome case detected.9 However, a decision to implement a particular screening programme should be based on as full an assessment of as many as possible of the relevant outcomes of a screening programme. In 1993, in the absence of clear regional or national guidance, Oxfordshire health authority had not decided whether to purchase a serum screening programme. In that same year about a third of pregnant women over 35 years in Oxfordshire chose to pay around £50 each to have biochemical tests for Down's syndrome performed privately; this choice was available only to those who could afford to pay. This was widely regarded as unsatisfactory. A decision was required about the NHS provision of serum screening for Down's syndrome for the women of Oxfordshire. The district health authority sought a screening option that was as effective or better at detecting Down's syndrome than current practice; addressed the controversy surrounding stress and the screening of younger women; and was cost neutral or cheaper in direct NHS costs than current practice.
We wanted to use the large amount of national and local data that is available about biochemical screening for Down's syndrome to quantify as many as possible of the likely consequences of different screening options for the population of Oxfordshire. We used the technique of decision analysis, which is a well documented method for integrating data from a wide variety of sources to explore population policy options10 11 and to help inform difficult decisions about individuals.12 The method consists of defining a clinical problem, identifying the components of the decision, arranging the components of the decision as a “tree” which describes the possible pathways to particular outcomes, and quantifying the probabilities of passing down each branch of the tree. This enables the expected consequences of different decisions to be calculated. It is also possible to use a decision tree to make quantitative estimates of the net value (utility) that would be expected to arise from different decisions by assigning a value to each of the different outcomes. Combining information about the likelihood of each outcome with the utility of that outcome allows the utility of each decision to be estimated and the “best” decision to be identified. This paper describes our analysis.
We considered six different screening programmes
Offering counselling, ultrasound, biochemical testing, and further counselling to all pregnant women regardless of age, with a “high risk” result defined as a predicted risk greater than 1 in 250;
Offering counselling, ultrasound, biochemical testing, and further counselling to all pregnant women regardless of age, with “high risk” defined as a predicted risk greater than 1 in 100;
Offering pregnant women aged 35 years or more amniocentesis or the option of biochemical testing for Down's syndrome paid for by the patient;
Offering amniocentesis to pregnant women aged 35 years and above;
Offering counselling, ultrasound, biochemical testing, and further counselling to pregnant women aged 30 years and above, with “high risk” defined as a predicted risk greater than 1 in 250; and
No screening programme for Down's syndrome.
Option 1 is a widely used policy for biochemical screening for Down's syndrome in the United Kingdom. Options 2 and 5 represent two different approaches to reducing the total numbers of false positive results, by increasing the threshold for labelling a pregnancy “high risk” (option 2) and by restricting the test to women aged 30 years or more (option 5). Option 3 represents existing practice in Oxfordshire, and option 4 is Oxfordshire's existing policy. The inclusion of option 6 (no screening for Down's syndrome) allows the overall population impact of the other policies to be determined.
We constructed a decision tree that described the different policy options and clinical courses of women's pregnancies from the second trimester. At each node or branching of the tree, probabilities or proportions were given to each branch by using published data or, when more appropriate, locally derived data. The tree was developed with local clinicians involved in antenatal care to ensure that it accurately described the options under consideration. The internal validity of the model was checked by comparing predictions made using the tree with observations collected for a locally conducted economic analysis.9 Once we were satisfied that the tree was a good summary of practice we used it to calculate the expected outcomes of different screening strategies. Sensitivity analyses were performed on every variable to identify which assumptions or values had the most influence on the outcomes of different screening strategies. The three most influential variables were then combined in a three way sensitivity analysis to determine the critical values at which the advantage of one policy over another would be lost. All calculations were performed using Smltree decision analysis software (J Hollonberg, 16B Pine North, Roslyn, NY 11576, USA).
Figure 1 shows the complete decision tree. Where branches are duplicates of others only one full branch has been displayed. Where appropriate, probabilities and values in sub-branches of the tree were calculated by using Bayes's theorem.
Seven sets of predicted population outcomes were calculated from the tree for each option: live birth with Down's syndrome; live birth without Down's syndrome; termination of a fetus with Down's syndrome; miscarriage attributable to amniocentesis; miscarriage with Down's syndrome; direct financial costs of the programme to the NHS; and the number of women offered counselling, ultrasound, biochemical testing, and further counselling. Table 1 shows the starting values used in the decision analysis.
The predictive validity of the tree and baseline data were checked by applying local population data to the tree and comparing the results with available real results. The predicted cost of screening all pregnant women in our district (population 548000) was £160000. Another district within the Oxford region (with a population of 269000) costed its screening programme for all women at £87000 for 1993-4. Our model predicted that 7.8 Down's births and 3.7 babies with Down's syndrome detected in pregnancies would be expected in Oxfordshire with present practice. In 1992 there were seven Down's syndrome births and four terminations for Down's syndrome in the John Radcliffe Hospital. Our model predicted an expected number of amniocenteses in Oxfordshire with present practice (population 548000) of 370 (6.8/10000). This compares with the total number of chromosomal analyses for Down's screening in the Oxford region (population 2580000) of 1700 in 1992 (6.6 per 10000).
Table II shows the predicted performance of each of the six screening policies and that different outcomes are optimised with different screening policies—for example, most fetuses with Down's syndrome were detected by offering counselling, ultrasound, biochemical testing, and further counselling to all pregnant women (option 1), but the number of normal births was greatest with no screening of any sort (option 6). The results also confirm that the numbers of Down's syndrome pregnancies detected could be increased and, simultaneously, the number of amniocenteses and associated miscarriages could be reduced by replacing existing practice with a screening option that includes serum testing.
Of the options that we considered, the one that best met the constraints of our district health authority purchasing team was the option in which pregnant women of 30 years and over are offered counselling, ultrasound, biochemical testing, and further counselling (option 5). This option detected more fetuses with Down's syndrome than current practice; did not offer screening to large numbers of young women and so avoided much of the controversy surrounding the psychological costs of screening; and was cheaper than current practice. It also resulted in a lower rate of loss of normal fetuses due to the screening programme itself and, if adopted, would offer the choice of a popular test to older women regardless of ability to pay. However, all the policy options for using a serum test that we have considered have important drawbacks: screening women aged > 30 years with high risk defined as > 1:250 (option 5) identifies fewer Down's syndrome pregnancies than screening the whole population with high risk defined as > 1:250 and would deny young women the opportunity of improving their chances of detecting a Down's pregnancy; screening the whole population with high risk defined as > 1:250 (option 1) leads to more miscarriages associated with amniocentesis and more false positive results and is more expensive than other options; and screening the whole population with high risk defined as > 1:100 (option 2) detects fewer cases of Down's syndrome than defining high risk as > 1:250 and is more expensive than present practice.
An alternative way of comparing the options is to consider the marginal costs and benefits of the different options. For example, comparison of options 1 (screening women of all ages) and 5 (screening women aged 30 years or more) reveals that screening pregnant women under 30 years of age as well as those aged 30 years or more would prevent the birth of approximately one extra baby with Down's syndrome each year. The costs to the population of preventing this one extra Down's birth would be one to two normal babies lost after amniocentesis; 4500 blood tests for young women (with the associated anxiety and counselling); 200 “high risk” results in young women and subsequent negative results on amniocentesis in all but one or two cases, with the associated stress and need for counselling; and £90000 for the extra tests, counselling, and amniocenteses. Deciding which of the options is “better” for the population depends on the relative importance attached to each of the outcomes.
The variables to which a decision to favour screening women over 30 years (option 5) over present practice (option 3) is most sensitive if the critical outcome is detection rate are the sensitivity of the serum test, the uptake rate for serum testing, and the acceptance rate of amniocentesis after a “high risk” serum test result. The variables to which the decision is most sensitive if the critical outcome is cost are the costs of a serum test, associated counselling, and ultrasound; the cost of amniocentesis and related laboratory work; and the specificity of the serum test. For each of these two outcomes (detection rate and cost) the three variables to which the decision is sensitive were combined in three way sensitivity analyses to determine the effect of altering the values of the three critical variables together. The results are shown in table III and figures 2 and 3. Figure2 shows the relation between the critical variables when serum screening for women aged 30 years or more is compared with present practice with respect to cost. Each separate line represents values of the relevant variables at which the programme costs of the two programmes under consideration are equal. To favour offering screening to women over 30 years of age, values must fall above and to the right of the line. Similarly, figure3 shows the values of the critical variables at which the two programmes would be expected to detect equal numbers of Down's pregnancies. Again, to favour screening women over 30 years of age, the real values must fall above and to the right of the line.
ADVANTAGES OF DECISION ANALYSIS
Decision analysis has allowed us to integrate the results of published data from various sources with local data uniquely relevant to the population for whom we have responsibility. This has several advantages.
Firstly, most published reports about Down's screening consider information about only one or two of the many relevant outcomes of a Down's syndrome screening programme, not all of which are beneficial. Recently much discussion has centred on the relative importance of various outcomes, especially loss of normal pregnancy13 and the psychological costs of screening.8 Decision analysis has enabled us to predict what would be expected to happen to a larger number of important population outcomes of different screening strategies than we were able to do from the literature alone.
Secondly, we were able to include details about the local population such as age specific fertility rates, age specific rates of Down's syndrome, and screening acceptance rates. We are thus as confident as we can be that our conclusions apply to our local population.
Thirdly, decision analysis has allowed the population consequences of different screening strategies to be communicated in such a way that the debate about screening can focus on the outcomes of the different possible programmes. The process also allows the assumption on which policy recommendations are based to be made explicit. If there is uncertainty about a particular value (for example, uptake rate or test sensitivity) the expected consequences for the screening programme can be estimated again. We found that the numbers about which there was most uncertainty had insufficient impact on the outcomes of the screening programme to alter the conclusions of the analysis.
LIMITATIONS OF DECISION ANALYSIS
The accuracy of the outcome predictions from the decision analysis depend on the accuracy of the decision tree and the validity of the data on which the calculations are based. Most of the data we used were published in peer reviewed journals. Local costs were taken from a recently published and locally performed economic analysis of serum screening.9 We included an estimate of direct costs to the health service and excluded the costs to others, including the costs of patient funded serum testing. We made no attempt to consider the wider economic impact of altering the number of children born with Down's syndrome as the purchasing health authority had indicated that it was unlikely to take these wider issues into account in reaching a decision about screening for Down's syndrome. The sensitivity analysis shows that offering counselling, ultrasound, biochemical testing, and further counselling to all pregnant women over 30 years of age would provide a cheaper and more effective screening service than current practice unless the specificity of the test were below 89%; the cost of the test, including counselling, were above £25; or the cost of amniocentesis were below £134. These critical values allow a large degree of uncertainty about the true costs of the tests before the reliability of the decision is called into question.
Similarly, local and national data2 14 suggest that screening programmes are likely to perform considerably better than the critical values of the key variables—that is, sensitivity of the serum test (above 43%), uptake of the test (above 64%), and uptake of amniocentesis after a “high risk” result on screening (above 70%). The value about which there is most uncertainty in this context is the added cost per biochemical test with ultrasound and counselling. The figure of £13.70 per patient that we used is the extra cost over and above the present cost of counselling, ultrasonography, and (alpha) fetoprotein testing in our district in 1993. This cost is set at zero in the policies that do not include counselling, ultrasound, biochemical testing, and further counselling paid for by the NHS. The validity of this costs data was examined and defended in the correspondence that followed publication of the cost effectiveness study from which our figures are derived.15 Estimates from centres in the United Kingdom that have started offering testing are around £15 to £25, including counselling; these costs would not affect our decision. However, there is still much debate about how much counselling is needed, particularly before blood is drawn, and prices in the private sector are sometimes nearer £50, which would make screening women over 30 years of age more expensive than present practice and screening of all women even more expensive.
Any decision tree is a simplification of reality. We have not considered other abnormalities that might be detected as a consequence of counselling, ultrasound, biochemical testing, and further counselling, such as Edward's syndrome, neural tube defects, multiple pregnancies, and intrauterine death. Equally, we have not considered the impact of ultrasound screening on the identification of Down's syndrome. However, our estimates of sensitivity and specificity of the serum test assume that an ultrasound scan has been performed in estimating gestational age. We applied overall sensitivity and specificity of serum testing to the whole population and not a series of age specific values. This was done to reduce complexity. The impact of this approximation is to underestimate the performance of serum testing in older women as, for a given risk cut off point, serum testing has a higher detection rate in older women, albeit at the expense of a higher false positive rate.16 17 In practice this might prompt a revision in the cut off risk level for a screening programme for older women.
USING DECISION ANALYSIS
Discussing the results locally, we found that opinions were divided about which, if any, of the options represents the best policy for Oxfordshire. People's judgments seem to hinge on two sets of issues: the relative values that they attach to the prevention of Down's births, iatrogenic loss of normal pregnancies, distress associated with false positive serum test results, and perceptions of the benefits of alternative uses for health service funds; and the relative weight they attach to the interests of the population as a whole compared to the interests of individuals.
Incorporating value judgments into the decision
There are two ways to incorporate value judgments into a decision analysis. The first is to use decision analysis to make the outcomes of different options explicit and then use the results to stimulate a debate about the relative advantages and disadvantages of the outcomes produced by the different options. This is the approach we have adopted because we believe that the values that different people attach to the different dimensions of outcome vary substantially and we wanted to avoid incorporating subjective value judgments into our results. Presenting the results as we have (table II) allows people to apply their own judgments to the different outcome packages and reach their own conclusions about which option they think offers the best combination of risk and benefits. Outcome data can stimulate debate about the relative importance of the different outcomes considered. Where people's values are very different then debate about outcomes and values, although useful in reaching an informed decision, can be tense.
An alternative approach would have been to make numerical assessments of the values (utilities) that relevant groups assign to different outcomes by using one or more of a number of standard methods for assessing utilities, such as the “standard gamble.” These values could then have been incorporated into the decision analysis, allowing the net utility of different options to be calculated. However, as different methods for measuring preferences can produce very different results, there is considerable doubt about the validity of existing methods for quantifying people's preferences—and hence of the validity of global utility measures.
Recognising both population and individual perspectives
The analysis presented here describes the expected population outcomes of various policies for screening for Down's syndrome. It shows that of the options that we have considered the one that best meets the purchasing authority's constraints is the policy of offering women over the age of 30 counselling, ultrasound, biochemical testing, and further counselling for Down's syndrome. This policy is better (in health and financial terms) than both the existing policy (offering amniocentesis to women over age 35) and the existing practice (a third of women over 35 buying a biochemical test). However, it is also important to consider the impact of health policy on individuals as well as on the population as a whole. In this case a major problem with the age based option is that it would deny individual young women access to an investigation that may be of benefit to them. Those who would be most disadvantaged would be young women to whom it matters more to avoid a Down's baby than it does to risk losing a potentially normal pregnancy and who also understand and accept the risk of a false positive result.
As society places a high value on not subjugating individual needs to those of the population, it may be inappropriate to use existing serum tests for Down's syndrome, with their relatively poor specificities and sensitivities, as population screening tools. Where the importance that individuals within the population attach to different outcomes varies widely, as seems likely for serum testing for Down's syndrome, the rigid application of any “one size fits all” population based policy would result in clinicians being unable to act in the best interests of each patient. It may therefore be better to use serum testing as an investigation that, like many other laboratory tests, is applied after the clinical circumstances and preferences of each individual have been assessed. However, insufficient data are available to be able to estimate the population impact of such a policy, and it is also unclear whether this would be either practical or affordable as, in practice, antenatal screening programmes are observed to have found it difficult to provide high quality counselling successfully to large numbers of women.18
The methods we have used may be of relevance beyond Oxfordshire. Decision analysis is a useful tool for bringing research findings to bear on policy making. It is also potentially a useful tool for incorporating individual patients' preferences into clinical decisions. It can be used to focus attention on the outcomes of screening programmes or individual interventions and the beliefs that people hold about what is important. Decision analysis has allowed us to quantify the expected population outcomes of various policies for screening for Down's syndrome and has focused local debate on the consequences of screening. In so doing it has made the debate accessible to a wider group of people. It has also meant that the drawbacks as well as the advantages of each of the options have been made more explicit. The analysis has identified an option that meets the predetermined constraints of the local purchasing authority—namely, offering women in Oxfordshire over the age of 30 years counselling, ultrasound, biochemical testing, and further counselling for Down's syndrome. Our local health authority is not the first to have considered an age based option,19 20 and at least one health authority has implemented one.3 However, were this policy to be implemented rigidly it would mean that young women to whom it matters more to avoid a Down's baby than it does to risk losing a potentially normal pregnancy would not be offered the care that gave them the best possible chance of achieving the outcomes they desired. This emphasises the importance of considering the impact of policy decisions on individuals as well as on the whole population.
If, as seems likely, the importance that individuals attach to different outcomes of serum testing for Down's syndrome varies widely, then the rigid application of a “one size fits all” screening policy may be inappropriate. If the needs of individuals are not to be subjugated to those of the population, serum testing for Down's syndrome should be an option for all pregnant women and its use determined by the individual preferences of well informed women. If the needs of the population are dominant then the choice of screening policy for Down's syndrome depends on the relative importance attached to the different outcomes of the screening programme.
We are grateful to Mrs Maggie Fitchett, director, Oxford Medical Genetics Laboratories, Churchill Hospital, Oxford Radcliffe NHS Trust, and to Mr James Raftery, health economist, Wessex Institute of Public Health, for help and advice and for comments on earlier drafts of this paper and to Ms Sue Standing, Department of Clinical Biochemistry, Oxford Radcliffe Hospital, for help in determining the costs of laboratory investigations.
Funding No additional funding.
Conflict of interest None.