Covid-19 testing, low prevalence and the impact of false positive results
Dear Editor,
We read with great interest the article on interpreting Covid-19 test results, namely the implications both false positive and false negative results may have on patients [1].
We are writing to share the analysis of positive results in our Health Board from 20/6/2020 to 21/7/2020 during a period of very low prevalence in this area. Over this period 31 throat swabs tested positive for SARS-CoV-2 by Real-time polymerase chain reaction (RT-PCR) on a number of platforms (Seegene (n=15), Cepheid (n=15), Luminex (n=1)). These platforms use a combination of targets including the N and E gene (Cepheid and Luminex) and the N, E and RdRp gene (Seegene).
Our results show that 26/31 were positive at low level (CT value >35) in a single gene and five were positive in more than one gene: three were positive in two genes (Cepheid n=2, Seegene n=1), and two positive in three genes (Seegene).
Of the 26 single gene low level positive results, 19 were repeated and all 19 were negative on repeat testing. Of these 26, 17 were asymptomatic and underwent screening for the following indications: pre-operative (5), nursing home residents (5), transplant waiting list (3), pre-discharge (2), pre-delivery (1), and nursing home key worker (1).
Nine of the 26 were symptomatic. The threshold values (Ct) in these patients ranged from 36 to 43, with a mean score of 39.5. One had a previous positive result and the result likely reflected residual genetic material. Five symptomatic individuals were re-tested and all re-tests were negative. Two were re-tested on the same day, one the following day, one after five days and one after eight days.
Five were positive in more than one gene. Two were asymptomatic pre-discharge patients. One was most likely a true positive case. He was asymptomatic at the time of swabbing but was symptomatic on admission with CXR changes and lymphopenia. His initial negative test at the time of admission was likely a false negative. The other asymptomatic individual was asymptomatic at the time of swabbing and was negative on repeat swabs on the same day and two days later. However, he was positive three months earlier and was antibody positive and so we think the positive result likely reflects the presence of residual RNA. Three were symptomatic. One was positive in two genes and two were positive in three genes. Overall, there were 12 symptomatic patients (nine were positive in a single gene, three in more than one gene).
Several potential significant implications for the single gene low level positive results were recognised. Patients on the transplant waiting list were removed from the list for two weeks. Some of the patients screened pre-operatively had their surgery delayed. Patients screened pre-discharge were kept in hospital, unnecessarily in many cases. All of the low level, likely false positive results from the nursing home residents and staff generated further activity including as a minimum re-swabbing but also triggering track and trace of residents and staff in some cases. There is also the potential for these results to negatively impact on staffing levels and transfers in and out of the home as well as causing a distraction from other elements of patient care. These adverse consequences of false positive results are proportionally more of an issue when prevalence is low.
Prevalence is a measure of how common a disease is in a specified at-risk population at a specific time point or period [2]. It measures the disease burden for the specified population [2]. Prevalence affects the pre-test probability of a disease being present and consequently impacts on the Positive Predictive Value (PPV) (the probability that subjects with a positive test truly have the disease) and the Negative Predictive Value (NPV) (the probability that subjects with a negative test truly do not have the disease) [3]. As the prevalence increases the PPV increases but the NPV decreases. Similarly, as the prevalence decreases the PPV decreases while the NPV increases [3]. The adverse outcomes associated with false positive results will be proportionally more significant during periods of low prevalence [3]. COVID19 provides a unique challenge because the prevalence of the disease is changing in real time and in line with prevention measures, principally the lockdown. This moving prevalence impacts on testing strategies and the interpretation of results. It also enables clinicians to witness the effects of prevalence and interpretation of results based on PPV and NPV in real time.
The harm afforded by false positive results should not be ignored and the potential for adverse consequences during periods of low prevalence needs to be taken into account when deciding on testing strategies. We recommend that testing strategies need to be more agile and decisions on screening of various populations should be flexible and respond to the changing prevalence in the community / setting that is being investigated. Large volume screening at a time of low prevalence has the potential to do more harm than good and some of these strategies should be temporarily suspended. Some of these strategies are likely to be of greater benefit in interrupting transmission during periods of high prevalence and we propose that they are re-instated when the prevalence in the community or particular settings warrant such an approach.
References
1. Watson, J, Whiting, P.F, Brush, J.E. Interpreting a covid-19 test result. BMJ. 2020;369(m1808) (May 11, 2020).
2. Tenny S, Hoffman MR. Prevalence. [Updated 2020 Apr 27]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2020 Jan-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK430867/
3. Parikh R, Mathai A, Parikh S, Chandra Sekhar G, Thomas R. Understanding and using sensitivity, specificity, and predictive values. Indian Journal of Ophthalmology. 2008;56(1):45.
Competing interests:
No competing interests
31 July 2020
Brendan Healy
Consultant in Microbiology and Infectious Diseases
Azizah Khan, Medical Student, Huria Metezai, Medical Student, Ian Blyth*, Hibo Asad*, Abi Holborow *, Claire Johnston*. *Public Health Wales Microbiology Department, Swansea
Public Health Wales Microbiology Department, Cardiff and Swansea
Public Health Wales Microbiology Department, 8 Eaton Cres, Sketty, Swansea SA2 8QA
Rapid Response:
Covid-19 testing, low prevalence and the impact of false positive results
Dear Editor,
We read with great interest the article on interpreting Covid-19 test results, namely the implications both false positive and false negative results may have on patients [1].
We are writing to share the analysis of positive results in our Health Board from 20/6/2020 to 21/7/2020 during a period of very low prevalence in this area. Over this period 31 throat swabs tested positive for SARS-CoV-2 by Real-time polymerase chain reaction (RT-PCR) on a number of platforms (Seegene (n=15), Cepheid (n=15), Luminex (n=1)). These platforms use a combination of targets including the N and E gene (Cepheid and Luminex) and the N, E and RdRp gene (Seegene).
Our results show that 26/31 were positive at low level (CT value >35) in a single gene and five were positive in more than one gene: three were positive in two genes (Cepheid n=2, Seegene n=1), and two positive in three genes (Seegene).
Of the 26 single gene low level positive results, 19 were repeated and all 19 were negative on repeat testing. Of these 26, 17 were asymptomatic and underwent screening for the following indications: pre-operative (5), nursing home residents (5), transplant waiting list (3), pre-discharge (2), pre-delivery (1), and nursing home key worker (1).
Nine of the 26 were symptomatic. The threshold values (Ct) in these patients ranged from 36 to 43, with a mean score of 39.5. One had a previous positive result and the result likely reflected residual genetic material. Five symptomatic individuals were re-tested and all re-tests were negative. Two were re-tested on the same day, one the following day, one after five days and one after eight days.
Five were positive in more than one gene. Two were asymptomatic pre-discharge patients. One was most likely a true positive case. He was asymptomatic at the time of swabbing but was symptomatic on admission with CXR changes and lymphopenia. His initial negative test at the time of admission was likely a false negative. The other asymptomatic individual was asymptomatic at the time of swabbing and was negative on repeat swabs on the same day and two days later. However, he was positive three months earlier and was antibody positive and so we think the positive result likely reflects the presence of residual RNA. Three were symptomatic. One was positive in two genes and two were positive in three genes. Overall, there were 12 symptomatic patients (nine were positive in a single gene, three in more than one gene).
Several potential significant implications for the single gene low level positive results were recognised. Patients on the transplant waiting list were removed from the list for two weeks. Some of the patients screened pre-operatively had their surgery delayed. Patients screened pre-discharge were kept in hospital, unnecessarily in many cases. All of the low level, likely false positive results from the nursing home residents and staff generated further activity including as a minimum re-swabbing but also triggering track and trace of residents and staff in some cases. There is also the potential for these results to negatively impact on staffing levels and transfers in and out of the home as well as causing a distraction from other elements of patient care. These adverse consequences of false positive results are proportionally more of an issue when prevalence is low.
Prevalence is a measure of how common a disease is in a specified at-risk population at a specific time point or period [2]. It measures the disease burden for the specified population [2]. Prevalence affects the pre-test probability of a disease being present and consequently impacts on the Positive Predictive Value (PPV) (the probability that subjects with a positive test truly have the disease) and the Negative Predictive Value (NPV) (the probability that subjects with a negative test truly do not have the disease) [3]. As the prevalence increases the PPV increases but the NPV decreases. Similarly, as the prevalence decreases the PPV decreases while the NPV increases [3]. The adverse outcomes associated with false positive results will be proportionally more significant during periods of low prevalence [3]. COVID19 provides a unique challenge because the prevalence of the disease is changing in real time and in line with prevention measures, principally the lockdown. This moving prevalence impacts on testing strategies and the interpretation of results. It also enables clinicians to witness the effects of prevalence and interpretation of results based on PPV and NPV in real time.
The harm afforded by false positive results should not be ignored and the potential for adverse consequences during periods of low prevalence needs to be taken into account when deciding on testing strategies. We recommend that testing strategies need to be more agile and decisions on screening of various populations should be flexible and respond to the changing prevalence in the community / setting that is being investigated. Large volume screening at a time of low prevalence has the potential to do more harm than good and some of these strategies should be temporarily suspended. Some of these strategies are likely to be of greater benefit in interrupting transmission during periods of high prevalence and we propose that they are re-instated when the prevalence in the community or particular settings warrant such an approach.
References
1. Watson, J, Whiting, P.F, Brush, J.E. Interpreting a covid-19 test result. BMJ. 2020;369(m1808) (May 11, 2020).
2. Tenny S, Hoffman MR. Prevalence. [Updated 2020 Apr 27]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2020 Jan-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK430867/
3. Parikh R, Mathai A, Parikh S, Chandra Sekhar G, Thomas R. Understanding and using sensitivity, specificity, and predictive values. Indian Journal of Ophthalmology. 2008;56(1):45.
Competing interests: No competing interests