Put to the test: use of rapid testing technologies for covid-19
BMJ 2021; 372 doi: https://doi.org/10.1136/bmj.n208 (Published 03 February 2021) Cite this as: BMJ 2021;372:n208Linked Feature
Covid-19: How the UK is using lateral flow tests in the pandemic
Read our latest coverage of the coronavirus outbreak
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor,
The Centers for Disease Control and Prevention (CDC) reported that compared to PCR testing, Abbott BinaxNOW had a sensitivity of 35.8% in asymptomatic people, rising to 64.2% in symptomatic people. It therefore missed about two-thirds of Covid-19 infections in asymptomatic people. Specificity (99.8%–100%) was high in both asymptomatic and symptomatic people.
Although the sensitivity of the BinaxNOW antigen testing to detect infection was lower compared with PCR testing, its sensitivity rose to 92.6% in symptomatic people and 78.6% in asymptomatic people using positive viral culture . CDC team said the finding "might reflect better performance for detecting infection in a person with infectious virus present." In this light, CDC recommended that "community testing strategies focused on preventing transmission using antigen testing should consider serial testing (e.g., in kindergarten through grade 12 schools, institutions of higher education, or congregate housing settings)".
The findings were reported in the January 1 issue of the CDC's Morbidity and Mortality Weekly Report (Ref).
My takeaway is that ART in a real-world setting was evidently less accurate than the current gold standard PCR test for screening asymptomatic people, and also less accurate than in the clinical performance on which emergency use authorization (EUA) was based. However, serial ART testing can detect infectious cases accurately and prevent outbreaks rapidly, making it a powerful tool in preventing transmission compared to a 24-hour turnaround PCR test.
Reference:
1. Evaluation of Abbott BinaxNOW Rapid Antigen Test for SARS-CoV-2 Infection at Two Community-Based Testing Sites — Pima County, Arizona, November 3–17, 2020
https://www.cdc.gov/mmwr/volumes/70/wr/mm7003e3.htm?s_cid=mm7003e3_w
Competing interests: No competing interests
Dear Editor,
A bad test is worse than no test
I am concerned to learn that Lateral Flow Tests are to be implemented in schools and homes. The roll-out looks set to bypass professional scrutiny for safety, reliability and ethics. The population is being invited to assume that the unsupervised repeated performance of these invasive procedures is somehow to benefit the individual and society. There appears to be no prior informed consent giving the usual caveats as to the individual costs/benefits of undertaking the test. There appears to be widespread belief, which is not being discouraged, that a positive lateral flow test automatically means that an asymptomatic individual may be spreading the virus.
As a doctor who assists a local Sports Club, regular and frequent PCR and Lateral Flow tests have become routine practice for many months. They have not assisted us in informing or improving our good biosecurity protocols. On the contrary, the amount of time taken to administer the tests is excessive, and wasteful of tight resources.
Echoing recent publications in BMJ and elsewhere, we found the lateral flow tests to be unreliable. Notwithstanding the predicted fall in sensitivity when mass testing an asymptomatic population, the problem with the tests is that they do not have the near 100% specificity that they demonstrated under clinical trial conditions when the tests are produced at scale and furthermore are used outwith the manufacturer’s directions at population level.
In fact we estimated within the first month of using lateral flow tests that we were seeing about 8 per thousand false positives in a population where the prevalence is low. We double checked all positives by repeat lateral flow tests and by PCR.
The reduction in specificity drastically reduces the Positive Predictive Value. See Table below.
Prevalence Sensitivity% Specificity% PPV% NPV% Comment
1 in 10 96.2 99.9 99.07 99.6 Pack Insert Data - Nasopharyngeal
1 in 4 96.7 99.8 99.3 99.02 Pack Insert Data – Nasal Swabs
1 in 100 96.2 99.9 90.7 99.96 Typical population prevalence
1 in 100 96.2 99 49.28 99.96 True estimated Specificity - ditto
1 in 100 78 99 44.07 99.8 Best operational sensitivity*- ditto
1 in 200 78 99 28 99,9 Prevalence w/e 20 Feb 2021
*Ref 1,2
Doctors have a duty to protect their patients, One such protection is contained within the GMC Code of Conduct, which requires clinicians to give a patient informed consent before an intervention, and in particular if it is an invasive procedure.
An example of a documented Informed Consent for asymptomatic testing using lateral flow devices may be as follows:
1. This test was developed using controlled conditions and in a symptomatic population.
2. In general use in an asymptomatic population, plus the variability inherent in administering self-tests, the sensitivity and specificity of the test may be affected. This means that the test’s reliability is reduced.
3. You must comply carefully with the directions for taking the test. A nasopharyngeal swab must be inserted through the nostril until it meets the back of the throat (about 4 inches) A nasal swab must be inserted into both nostrils until it is at the bridge of the nose (about 2 inches).
4. Interpreting the Test Results: A positive test result means that there is a 50% chance or less that you are infectious. A negative test means that you are almost certainly not infectious.
5. A positive lateral flow test can no longer be checked by requesting an NHS PCR. Therefore if you get a positive test you must self-isolate for ten days and ensure that your family and contacts also do so.
6. Please note that lateral flow tests are not sufficient to determine whether a contact is or is not infected.
7. Your doctor or nurse would be unable to advise on an individual lateral flow test taken in the community. Please follow Government Guidelines.
Prevalence of Covid19 infections last week was reported as about one in 200. Due to the unreliability of the tests to predict whether this is true or not (Positive Predictive Value) where prevalence is low, you would be better off tossing a coin to find out whether you are infectious and therefore a risk to society.
References:
1 . Torjesen I. How the UK is using Lateral Flow Tests in the Pandemic. BMJ 2021; 372 :n287
https://doi.org/10.1136/bmj.n287 (Published 04 February 2021)
2. . Lam K, Rapid Response to: How the UK is using Lateral Flow Tests in the Pandemic
https://www.bmj.com/content/372/bmj.n287/rr-0
3. Crozier, A. Put to the test: use of rapid testing technologies for covid-19 BMJ 2021; 372 :n208
https://doi.org/10.1136/bmj.n208 (Published 03 February 2021)
4 . Lewis T. Rapid Response to: Put to the test: use of rapid testing technologies for covid-19
https://www.bmj.com/content/372/bmj.n208/rr
Competing interests: No competing interests
Dear Editor,
I refer to Figure 1 entitled "High frequency testing with low analytic sensitivity versus low frequency testing with high analytic sensitivity".
A picture tells a thousand words and the message is that frequency of testing is vastly more important than test sensitivity. Let me use the graph to go through the dynamics of Covid-19 transmission. when a person first becomes infected, there will be an incubation period when no test will be able to detect the viral infection, because the viral load is so low. A PCR test can detect the virus up to 10^3 cp/ml while a rapid antigen test can only detect up to 10^5 cp/ml. On the third day, the viral load is rising rapidly which can be detected by PCR test only. Beyond that point, even if a rapid antigen test is 1,000 times less sensitive than a PCR test, the virus is increasing exponentially that the test will turn positive within 24 hours. Also the window of infectivity is so narrow which is about 7 days. Using a highly sensitive but expensive and time consuming PCR test, a cheap (<$5)rapid antigen test which is used frequency can capture most infections while they are still infectious.
Assuming the frequency of testing is 3 tests per week. Therefore, the probability of detecting Covid-19 can be computed statistically in a serial testing using Innova LTF's 73% sensitivity when used by trained healthcare staff as follows.
1- (1-sensitivity)^3 = 1- (1-0.73)^3 = 98%
Therefore, frequent use of the inferior test such as Innova LFT can enhance it sensitivity comparable to PCR test. It is time to change how we think about the sensitivity of mass testing for Covid-19 to break the chain of transmission.
Competing interests: No competing interests
Dear Editor
This paper provides a useful summary of the where we are with lateral flow testing (LFT) for SARS-CoV2 infection, and different ways in which it can be used. The prior probability that an individual has SARS-CoV2 infection is of fundamental importance. If 75% of infections are symptomatic then the probability that an asymptomatic person has covid is no more than 0.1%, even in high prevalence areas. In our experience, staff working in outbreak areas, but without declared symptoms, have a prior risk of infection of about 3%. Symptomatic staff (using a broader definition of symptomatic, including myalgia, headache, sore throat) have a prior risk of about 8%.
LFTs may have a significant utility in these higher prior probability situations, where the positive predictive value (PPV) will be about 95%, with a negative predictive value (NPV) around 97-99%. For instance, they may allow more proactive management of those identified as contacts with covid (“test to release”) or those living or working in a localised outbreak setting (“cluster response testing”). Alternatively, LFTs might have utility in the management of people with relatively non-specific symptoms that have a low odds ratio for predicting covid. The ultimate risk assessment here would depend on the presence of additional control measures (eg. PPE), the consequences of false reassurance (eg. Mixing with vulnerable people), and the possibilities for deferring any activity.
It has consistently been shown in the literature, and is also our experience from detailed local contact tracing, that people without symptoms are not likely to contribute significantly to transmission (Cevik et al, doi: https://doi.org/10.1136/bmj.m3862). Although LFTs have a very high NPV in this situation, it is doubtful that they add much to a good history in this context. “Being wrong” about people with no symptoms is both unlikely, and of low consequence.
On the flip side, the PPV of tests in this setting will be around 50%. This is supported by our local findings where 10 out of 20 positive LFTs performed in the last month in staff (a period of near zero nosocomial infection) have not been confirmed by subsequent PCR. Thus, around 1 in every 500 LFTs performed might be expected to give a false positive result. This harm will be direct to the affected individuals (eg quarantine, mis-classification of future risk), but also gives a potentially false impression that effective action is being taken. This misdirects us from the key problems of how to effectively identify symptomatic cases (using a broader definition of symptomatic than is currently used in official guidance), how we identify contacts in a timely fashion, and then, crucially, how we support these people to isolate.
In short, LFTs are a potentially useful part of our management of the pandemic. It is crucial we learn how to direct their use to maximum effect. It is not clear that current strategies will achieve this.
Competing interests: No competing interests
Positive results from UK single gene PCR testing for SARS-COV-2 may be inconclusive, negative or detecting past infections
Dear Editor,
The efficacy of mass population testing for SARS-COV-2 virus is critically dependent on the reliability of the test applied, whether it be a RT-PCR or lateral flow test. Given that many RT-PCR tests do not actually target all the genes necessary to reliably detect SARS-COV-2, the results of mass testing using RT-PCR need to be revisited and reanalysed.
The ONS publish a regular infection survey [1], [15] that includes data from two UK lighthouse laboratories, where both use the same RT-PCR test kit to detect the SARS-COV-2 virus. This survey includes data on the cycle threshold (Ct) used to detect positive samples, the percentage of positive test results arising from using RT-PCR, and the combinations of the SARS-COV-2 virus target genes tested that gave rise to positives between 21 September 2020 and 1 March 2021 across the whole of the UK.
The kit used by the Glasgow and Milton Keynes lighthouse laboratories is the ThermoFisher TaqPath RT-PCR which tests for the presence of three target genes from SARS-COV-2 [2]. Despite Corman et al [3] originating the use of PCR testing for SARS-COV-2 genes there is no agreed international standard for SARS-COV-2 testing. Instead, the World Health Organisation (WHO) leaves it up to the manufacturer to determine what genes to use and instructs end users to adhere to the manufacturer instructions for use (IFU).
The WHO’s emergency use assessment (EUA) for the ThermoFisher TaqPath kit [4] includes the instruction manual and contained therein (Table 6) is an interpretation algorithm describing an unequivocal requirement that two or more target genes be detected before a positive result can be declared. The latest revision of ThermoFisher’s instruction manual contains the same algorithm [14]. The WHO have been sufficiently concerned about correct use of RT-PCR kits that on 20 January 2021 they issued a notice for PCR users imploring them to review manufacturer IFUs carefully and adhere to them fully [5].
The ONS’s report [1] lists SARS-COV-2 positive results for valid two and three target gene combinations and does the same in [15], for samples processed by the Glasgow and Milton Keynes lighthouse laboratories. However, it also lists single gene detections as positive results (See tables 6a and 6b). Over the period reported the maximum weekly percentage of positives on a single gene is 38% for the whole of the UK for the week of 1 February. The overall UK average was 23%. The maximum percentage reported is 65%, in East England in the week beginning 5 October. In Wales it was 50%, in Northern Ireland it is 55% and in Scotland it was 56%. The full data including averages and maxima/minima are given in [17].
Professor Alan McNally, Director of the University of Birmingham Turnkey laboratory, who helped set up the Milton Keynes lighthouse laboratory, contradicted what was stated in the ONS report in a Guardian newspaper article about the new variant. He reported that all lighthouse laboratories operated a policy that adhered to the manufacturer instructions for use: requiring two-or-more genes for positive detection [6] (this policy is also documented in [16], which defines the standard operating procedure reported in [7]).
In correspondence with Mr Nicholas Lewis about single gene testing, in February 2021, the ONS confirmed that they do indeed call single gene targets as positives in their Covid-19 Infection Survey and also confirmed that the samples are processed by UK lighthouse laboratories [8], [9].
As early as April 2020, the UK lighthouse laboratories were testing for single genes and discounted the S gene as early as mid-May [10], months before the discovery of the new variant B1.1.7. Indeed, in Table 1 of [10] 18% of tests were positive on one gene only. Furthermore, in a Public Health England report on variants [2] published 8 January 2021, it states the goal of using one gene was explicitly to approximate the growth of the new B1.1.7 variant:
“There has recently been an increase in the percentage of positive cases where only the ORF1ab- and N-genes were found and a decrease in the percentage of cases with all three genes. We can use this information to approximate the growth of the new variant.”
Section 10 of this ONS Covid-19 Infection Survey report [11] on 8 January 2021 confirms that one gene is sufficient for a positive result.
Obviously, there is a higher risk of encountering false positives when testing for single genes alone, because of the possibility of cross-reactivity with other human coronaviruses (HCOVs) as well as prevalent bacteria or reagent contamination. The potential for cross reactivity has already been confirmed by the German Instand laboratory report from April 2020 [12] (note that Prof. Drosten, co-author of Corman et al [2] is a cooperating partner listed in this report). The report describes the systematic blind testing of positive and negative samples anonymously sent to 463 laboratories from 36 countries and evaluated for the presence of a variety of genes associated with SARS-COV-2. They reported significant cross reactivity and resultant false positives for OC43, and HCoV 229E (a common cold virus) as well as for SARS-COV-2 negative samples, not containing any competing pathogen. Likewise, 70 Dutch laboratories were surveyed in November 2020, by the National Institute for Public Health and the Environment [13], with 76 diagnostic workflows reported as using only one target gene to diagnose the presence of SARS-COV-2 (46% of all workflows).
Without diagnostic validation it is not clear what can be concluded from a positive PCR test resulting from a single target gene call, especially if there was no confirmatory testing. Many of the reported positive results may be inconclusive, negative or from people who suffered past infection for SARS-COV-2. Even with diagnostic validation of the single target gene call, the UK lighthouse laboratories appear not to be in strict conformance with the WHO emergency use assessment and the manufacturer instructions for use. Given this it is clear the ONS and the UK lighthouse laboratories needs to publicly clarify their use of, and justify the reasons for, deviating from these standards.
References
[1] Steel K. and Fordham E. Office for National Statistics. Coronavirus (Covid-19) Infection Survey. 5 December 2020 (See tables 6a and 6b).
https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/...
[2] Public Health England “Investigation of novel SARS-COV-2 variant. Variant of concern.”, 202012/01.
https://assets.publishing.service.gov.uk/government/uploads/system/uploa...
[3] Corman V., Landt O. et al “Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR” Euro Surveillance. 2020 Jan;25(3):2000045. doi: 10.2807/1560-7917.ES.2020.25.3.2000045.
[4] WHO Emergency Use Assessment Coronavirus disease (COVID-19) IVDs. PUBLIC REPORT. Product: TaqPath COVID 19 CE IVD RT PCR Kit. EUL Number: EUL-0525-156-00. Page 60
https://www.who.int/diagnostics_laboratory/eual/200921_final_pqpr_eul_05...
[5] WHO Information Notice for IVD Users 2020/05. Nucleic acid testing (NAT) technologies that use polymerase chain reaction (PCR) for detection of SARS-CoV-2. 20 January 2021
https://www.who.int/news/item/20-01-2021-who-information-notice-for-ivd-...
[6] Alan McNally. “It's vital we act now to suppress the new coronavirus variant” Opinion section the Guardian Newspaper, 22 Dec 2020. https://amp.theguardian.com/commentisfree/2020/dec/22/new-coronavirus-va...
[7] Richter, A., Plant, T., Kidd, M. et al. How to establish an academic SARS-CoV-2 testing laboratory. Nat Microbiol 5, 1452–1454 (2020). https://doi.org/10.1038/s41564-020-00818-3
[8] Dr John Allen, ONS. Email correspondence to information request from Mr Nicholas Lewis, “Your ad hoc Covid-19 PCR gene detection analysis for the ONS”, 22 February 2021.
[9] Zoe (?), ONS. Email correspondence to information request from Mr Nicholas Lewis, ONS, email correspondence to information request from Mr Nicholas Lewis, “ONS ad hoc Covid-19 PCR gene detection analysis”, 25 February 2021.
[10] Walker S. Pritchard E et al. Viral load in community SARS-CoV-2 cases varies widely and temporally. https://www.medrxiv.org/content/10.1101/2020.10.25.20219048v1
[11] ONS Coronavirus (COVID-19) Infection Survey, UK: 8 January 2021. ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/8january2021#the-percentage-of-those-testing-positive-who-are-compatible-for-the-new-uk-variant
[12] Zeichhardt H., and Kammel M. “Comment on the Extra ring test Group 340 SARS-Cov-2” Herausgegeben von: INSTAND Gesellschaft zur Förderung der Qualitätssicherung in medizinischen Laboratorien e.V. (INSTAND Society for the Promotion of Quality Assurance in Medical Laboratories e.V.) 3rd June 2020.
https://www.instand-ev.de/System/rv-files/340%20DE%20SARS-CoV-2%20Genom%...
[13] External Quality Assessment of laboratories Performing SARS-CoV-2 Diagnostics for the Dutch Population. National Institute for Public Health and the Environment, Ministry of Health, Welfare and Sport., November 2020.
https://www.rivm.nl/sites/default/files/2021-02/EQA%2520of%2520Laborator...
[14] TaqPath COVID-19 Combo Kit and TaqPath COVID 19 Combo Kit Advanced INSTRUCTIONS FOR USE. Revision J.0, 22 February 2021. (See Table 25, page 107). https://assets.thermofisher.com/TFS-Assets/LSG/manuals/MAN0019181_TaqPat...
[15] Walker, S. 21 December 2020. Covid-19 infection Survey: Ct values analysis (Glasgow and Milton Keynes identified in Table 4a) https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/...
[16] Clinical Immunology Service, University of Birmingham. ‘Competency Assessment: Reporting, Interpretation and Authorisation of Results in Turnkey Birmingham’. CIS/TK44, v1.0. September 2020.
[17] Neil M. Positive results from UK single gene testing for SARS-COV-2 may be inconclusive, negative or detecting past infections. https://arxiv.org/abs/2102.11612
Competing interests: No competing interests