Testing for antibodies to SARS-CoV-2

Dear Editor,

We write in response to the BMJ article by Mulchandani et al (1) and the associated editorial which references an online pre-print of our work (2). Like Mulchandani et al, we note the difficulties in evaluating new assays designed to detect antibodies to SARS-CoV-2 virus due to the lack of a gold standard reference test to evaluate against. Statistically unbiased estimation of test sensitivity (the ability of the test to correctly identify true positives i.e. the rate of false negatives) is at present difficult. However, we caution that a direct comparison between the studies cannot be made, given there are clear differences between usage target and reference standards used.

In our pre-print (2) we report longevity of SARS-CoV-2 antibodies for over 20 weeks from time of symptoms or PCR positive status and we describe a laboratory evaluation of the AbC-19 test. In our evaluation of AbC-19 the target condition was antibodies – specifically SARS-CoV-2 Spike protein IgG antibodies. We evaluated the performance characteristics following guidance in the MHRA (3) target product profile which stipulates use of at least 200 samples confirmed as positive and at least 200 samples confirmed as negative. Samples for cohort inclusion were characterised by three available SARS-CoV-2 laboratory-based immunoassays (Roche Elecsys, Abbott Architect SARS-CoV-2 IgG and EuroImmun IgG ELISA) to determine samples as antibody positive or antibody negative. To create a positive cohort we do apply a reference standard which required a positive result in two of three assays; Euroimmun ELISA and one other of Roche or Abbott.

We note Mulchandani et al uses PCR positive status or Roche positivity, whilst they also model two immunoassays together (Roche and Euroimmun ELISA) as a composite reference standard. Roche detects IgG, IgM & IgA antibodies to the nucleocapsid portion of the virus. We prioritise Euroimmun ELISA (IgG to spike protein) alongside either Abbott or Roche positivity to evaluate AbC-19 which detects IgG antibodies to the full trimeric spike protein of SARS-CoV-2; the exact immunogenic protein of the SARS-Cov-2 virus that is used in global vaccines under development. The AbC-19 test could be used for assessing antibody responses to vaccination programmes in our battle against COVID-19.

We acknowledge statistically unbiased estimation of test sensitivity is difficult without an accurate and applicable single gold standard. We use antibody positive as an alternative to RT-PCR positive status as a standard for assessing SARS-CoV-2 antibody assays as our target condition is SARS-CoV-2 IgG antibodies. False positive PCR results may occur in the UK at a rate between 0.8- 4.0% (4) and false negative rates of RT-PCR in up to 30% (5). Furthermore, lack of detectable IgG (up to 90 days post symptom onset) was demonstrated in approximately one in 16 individuals despite previous RT-PCR confirmed infection (6).

The details of our analysis were originally described in a pre-print article (medRxiv) describing the performance of this test to detect antibodies across 654 samples. The cohorts were originally characterised as-

• Negative for SARS-CoV-2 IgG antibody- 350 negative by all three immunoassays- Euroimmun ELISA, Roche and Abbott.
• Positive for SARS-CoV-2 IgG antibody- 304 positive for IgG antibody to spike protein by Euroimmun ELISA and one other of Roche or Abbott.
The performance metrics of AbC-19 lateral flow test was reported against these characterised samples in a laboratory setting as 100% specificity and 97.7% sensitivity.

In the most recently updated medRxiv pre-print article (2) we extended the cohort and tested additional samples to include 818 samples with positive and negative cohorts characterised as-

• (1a) Negative Cohort- 223 pre-pandemic samples were assumed negative regardless of how they were characterised by all three immunoassays- Euroimmun ELISA, Roche and Abbott.
• (1b) Negative Cohort- 265 known negatives for SARS-CoV-2 IgG antibody- were negative by all three immunoassays-Euroimmun ELISA, Roche and Abbott.
• (2) Positive for SARS-CoV-2 IgG antibody- 330 were positive for IgG antibody to SARS-CoV-2 spike protein by Euroimmun ELISA and one other of Roche or Abbott.
The performance metrics of AbC-19 for the detection of antibodies in negative cohort 1a+1b (n=488) in a laboratory setting is 99.59% specificity and 97.58% sensitivity. These metrics were calculated without extrapolating PPV’s to a set prevalence given the number of assumptions that would be required to be implemented and justified.

Using only the pre-pandemic negative cohort 1a (n=223), a robust approach assuming negative antibody status regardless of characterisation on any other immunoassays used for the reference standard, and addressing the concern of bias, had minimum impact on performance metrics showing a 99.55% specificity.

It is important to stress we do not assess AbC-19 for its utility in detecting previous infection, our target condition is presence of antibodies. Our study also assessed analytical specificity of AbC-19 reviewing 34 samples from other known respiratory viruses confirming no cross-reactivity. We highlight faint T lines may be reflective of the longer time from infection for some of the Northern Ireland cohort used (up to 20 weeks post infection) and if AbC-19 is to be used in clinical settings it is important to determine if all users observe the same results as observed in this laboratory evaluation. We highlight high-throughput laboratory evaluation of thousands of samples, even when performed in batches, may result in sample drying on the AbC-19 test leading to incorrect read outs. Additional studies in different use cases with different users are needed depending on application.

We welcome further clinical evaluation of AbC-19 in large cohorts of symptomatic and asymptomatic individuals alongside larger longitudinal studies assessing COVID-19 outcomes in individuals to fully validate its implementation across all intended use cases.

Authors have not been paid or financially benefitted from this study. Advisory CIGA Healthcare roles were unpaid temporary roles. The authors clarify consumable only costs (assays and laboratory expenses) AbC-19 laboratory evaluation are to be claimed from UK-RTC as is normal practice. The manuscript and associated data have only been used to build confidence into the overall device design and performance assessment of AbC-19 and such work was never commissioned for any government contractual consideration.

1. Mulchandani R, Jones H, Taylor-Phillips S, et al., EDSAB-HOME and COMPARE Investigators Accuracy of UK Rapid Test Consortium (UK-RTC) “AbC-19 Rapid Test” for detection of previous SARS-CoV-2 infection in key workers: test accuracy study. BMJ2020;371:m4262

2. Robertson et al. SARS-CoV-2 antibody testing in a UK population: detectable IgG for up to 20 weeks post infection. https://www.medrxiv.org/content/10.1101/2020.09.29.20201509v1 doi: https://doi.org/10.1101/2020.09.29.20201509

3. Target Product Profile: antibody tests to help determine if people have recent infection to SARS-CoV-2: Version 2 (Updated 15 October 2020) https://www.gov.uk/government/publications/how-tests-and-testing-kits-fo.... Accessed 12 November 2020.

4. Surkova E, Nikolayevskyy V, Drobniewski F. False-positive COVID-19 results: hidden problems and costs. Lancet Respir Med [Internet]. 2020 Nov 11; Available from: https://doi.org/10.1016/S2213-2600(20)30453-7

5. Watson J, Richter A, Deeks J. Testing for SARS-CoV-2 antibodies. BMJ [Internet]. 2020;370. Available from: https://www.bmj.com/content/370/bmj.m3325

6. Petersen LR, Sami S, Vuong N, Pathela P, Weiss D, Morgenthau BM, et al. Lack of antibodies to SARS-CoV-2 in a large cohort of previously infected persons. Clin Infect Dis [Internet]. 2020 Nov 4; Available from: https://doi.org/10.1093/cid/ciaa1685