Assessment of changes to screening programmes: why randomisation is importantBMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h1566 (Published 30 March 2015) Cite this as: BMJ 2015;350:h1566
- Katy J L Bell, NHMRC early career research fellow1,
- Patrick Bossuyt, professor of clinical epidemiology2,
- Paul Glasziou, director1,
- Les Irwig, professor of epidemiology3
- 1Centre for Research in Evidence Based Practice, Bond University, QLD 4229, Australia
- 2AMC, Amsterdam, Netherlands
- 3Screening and Test Evaluation Program, School of Public Health, University of Sydney, Australia
- Correspondence to: K J L Bell
- Accepted 17 February 2015
Randomised controlled trials provide the best evidence of the benefits and harms of screening. Yet, once a screening programme is started, uncertainty may remain about specific components of the strategy. For example, what test should we use to screen for colorectal cancer? How long should we wait before re-screening? Should potential participants be given an information booklet? The screening technology itself is continually evolving and often there is uncertainty about to whether to introduce new testing methods, such as liquid based cytology for cervical cancer1 or 3D mammography to screen for breast cancer.2
We believe that such decisions should be based on the strongest available evidence. This requires pragmatic trials within the screening programme, with the eligible population randomised to different screening strategies. Here, we use the example of bowel cancer screening to show why randomisation is needed and how it can work. However, the general principles can be applied to population screening for other conditions.
Although randomised controlled trials show that offering faecal occult blood testing prevents death from colorectal cancer, there is little evidence about which test should be used. Most trials used guaiac faecal occult blood tests, but since then several immunochemical tests have become available and are claimed to have better performance. There have also been suggestions that the threshold for diagnosis should be modified according to previous risk and that the screening interval be based on cancer risk .3 Randomisation within screening programmes can inform these decisions and help prevent problems, as the experience of the Australian screening programme shows.
Australia’s screening programme for bowel cancer started in August 2006 using an immunochemical faecal test (Hemtube, Fujirebio, Japan). An upgraded version of the test was introduced in December 2008 (New HemTube), which was thought to have a longer shelf life. However, the programme experienced a big setback when it was realised that the new kits contained a faulty solution that meant red blood cells broke down in warm weather. Faulty kits were sent out for six months before the high negative rate was detected. Over 475 000 people received a faulty test, 4444 of whom had a positive result on retesting. Screening was suspended for a further six months before restarting with an updated and more stable kit. However, specialists remained concerned about the performance of the new test,4 and participation rates were significantly lower than before the stoppage, suggesting loss of public confidence.5
These events may have been at least partially avoided if decision makers had continued the methods of the pilot implementation programme. During the pilot, which ran from November 2002 to June 2004 in three sites, households were randomised to receive one of two immunochemical faecal occult blood home tests (Inform (Enterix) or Detect (Bayer)). Although both tests were done at home by participants and sent to laboratories for analysis, they differed in both the way the patient performed the test and chemical make-up. The randomisation in the pilot programme allowed process and intermediate outcomes to be compared for the two tests (table⇓).6
If randomisation in screening had been extended beyond the pilot it would have allowed continued comparison of these process outcomes. Problems with the Hemtube test would probably have been detected more quickly, and all participants could have been offered the alternative test while the reasons for malfunction were determined.
Continued randomisation within the population screening programme would have generated data from a much larger number of participants and for longer follow-up times. This would increase the power of analyses to detect clinically relevant outcomes such as differences in the number of cancers and advanced adenomas detected and in interval cancer rates. Offering annual versus biennial screening might also have been evaluated, along with other possible modifications to the screening strategies based on new and existing evidence.
Pilot studies of bowel cancer screening in the Netherlands also used randomisation to compare outcomes with guaiac and immunochemical tests7 8 9 and of once only screening with repeated screening at one, two, or three years.10 A national screening programme is now being introduced over five years using a different immunochemical test from that used in the pilot. The test was selected after public tender, but concerns have been expressed that it may be less sensitive than the piloted test.11 As a result, the Dutch government has funded a randomised comparison to provide evidence on the relative performance of the two tests. Although this is a welcome step, randomisation could have been incorporated from the start of the programme and needs to be sustained to maximise the benefits. Long term randomisations will allow comparison of clinically relevant benefits and harms rather than intermediate measures such as test positive and test negative rates.
Benefits of randomising in population screening
Randomisation in population screening programmes allows collection of unbiased evidence about comparative benefits of different strategies. No separate trials are needed, and no invitees disadvantaged.
Best way of dealing with uncertainty
Just as randomisation is the most fair and ethical way of dealing with the uncertainty of benefits and harms of new and established12 treatments, it is also the most fair and ethical way of dealing with uncertainty surrounding different screening strategies. The uncertainty about the best screening strategy is ethically dealt with by randomising participants to alternative screening strategies during pilot studies, implementation phase, and in the longer term. By providing screening invitees with information on what is uncertain about the best way to screen and how randomisation helps answer this, participants are able to provide informed consent and the whole process is transparent.
Provides best information on outcomes
Short term randomisation of different strategies within a screening programme allows evaluation using process measures and intermediate outcomes, such as participation, acceptability, positivity rate, yield (detection rate), and type or stage of lesions. Randomisation ensures that any observed differences are not due to differences in the people who have one type of screening strategy rather than another. Although process measures are useful for eliminating some strategies—for example, those with limited acceptability, a low participation rate, or a very low yield—we need more evidence before we can say that a particular strategy has higher real benefit. For example, estimates of the amount of screen detected disease in a population (yield) don’t differentiate between cases that would progress without treatment and cases that would remain clinically silent. Early detection of progressive disease is a benefit of screening whereas detection and treatment of clinically silent disease is an important harm.13
Evidence of benefit may come from showing lower morbidity or disease specific mortality (from early treatment of progressive disease). Evidence of harm may come from demonstrating a higher incidence (and treatment) of non-progressive disease (estimated from incidence after adjustment for lead time from the early detection of progressive cancers). However, even the large numbers of people in screening programmes may not be sufficient to detect subtle differences in mortality and morbidity, and randomised trials using these endpoints also require long follow-up periods. Proxy outcomes are therefore often used to compare the real benefits and harms of screening.
In cancer screening, a good proxy outcome for cancer specific mortality is the interval cancer rate—those that present clinically in the interval in between scheduled screening times after an initial negative result.14 As with all cancers which present clinically, interval cancers are likely to progress without early treatment. A difference in the interval cancer rate is apparent over a much shorter time frame than a difference in cancer specific mortality. Randomised controlled trials in screening programmes may still have insufficient power to detect a small (but clinically important) difference in interval cancer rate or to demonstrate equivalence with good precision. Nevertheless, they are better than the other options: relying on a smaller randomised controlled trial or unrandomised comparison of different population screening programmes, or not including data on interval cancer rate.
Safest way to introduce or change programmes
Randomisation to routine screening tests allows for the early identification of problems with one of the tests during implementation and in the longer term. Comparison of process and interval outcomes between the tests can flag up problems that require further investigation, adding to conventional quality control methods.
Once sufficient data are available, analysis of clinically relevant outcomes can show which is the best test. Similarly, if new knowledge or technology suggests that a screening programme should be changed, introducing this in a randomised way will show whether theoretical advantages are real. Using two or more types of screening test also maintains the competition between manufacturers for both quality and cost and means programmes are not reliant on a single manufacturer for supply.
Overcoming obstacles to randomisation
It may be argued that randomisation within screening programmes does not have sufficient benefit for participants and may confuse invitees, leading to low participation and limited health benefits. We believe that the alternatives offered should be sufficiently similar to limit these potential downsides—for example, two different tests, thresholds, or screening intervals. Radically different alternatives should still be evaluated in a traditional trial.
Consent for randomisation in population screening can be included in the invitation to be screened, which would mention the uncertainties about the best screening strategy and describe the different strategies used in the programme and that they are allocated randomly. Participants would be provided with information on how the alternatives differ, the processes for de-identifying patient data, and how data will be used to improve the screening programme.
Towards routine randomisation
Using multiple strategies adds complexity to screening, but we believe this is more than offset by the clear advantages outlined above. The Australian and Dutch examples show that it can be done if there is the political will, and that communities readily accept the randomisation. Processes need to be developed for working up alternative screening strategies, and the stages that this might go through, with consideration of when and how interim analyses should be done and the thresholds that might be used for deciding that one strategy is better than the other. With evidence that randomising in population screening is both feasible and beneficial, we believe that it should become more widely adopted within new and existing screening programmes.
Randomised trials within screening programmes provide the best answers to clinical uncertainty
Randomisation can be done during pilot studies, implementation, and in the longer term
Australian and Dutch examples show randomisation is feasible and acceptable to screening participants
Randomisation should be more widely used to assess real benefits and harms of screening and ensure best choice of screening strategy
Cite this as: BMJ 2015;350:h1566
We thank Graeme Young for his helpful comments on an earlier draft.
Contributors and sources: KJLB is an early career researcher at Bond University and University of Sydney. LI leads the Screening and Diagnostic Test Evaluation Program at University of Sydney. LI conceived the idea and developed them with all coauthors. KJLB wrote the first draft, edited the paper, and is guarantor. PB developed the ideas, edited the paper, and suggested Dutch data sources. PG developed the ideas and edited the paper, and suggested Australian data sources.
Competing interests: We have read and understood BMJ policy on declaration of interests and declare we have received funding from the Australian National Health and Medical Research Council.
Provenance and peer review: Not commissioned; externally peer reviewed.