Covid-19: “Unacceptable” that antibody test claims cannot be scrutinised, say experts

Independent evaluation

Roche’s test was the first to be approved, on 7 May. John Newton, national coordinator of the UK Coronavirus Testing Programme, said an independent evaluation carried out at the PHE Porton Down centre reported that the test was a “highly specific assay with specificity of 100%.”

Speaking at the daily briefing on 14 May, deputy chief medical officer Jonathan Van Tam said, “We’ve been waiting for a really good antibody test to be ready. There are now at least two available. One of those, the Roche test, received its CE mark on 28 April. It was validated by PHE on 7 May and I anticipate that it will be rapidly rolled out in the days and weeks to come as soon as it is practical to do so. I also anticipate that the focus will be on the NHS and on carers in the first instance.”

This was shortly followed by an announcement on Abbott’s test, which was released on 15 May. A spokesperson for Abbott said, “Abbott’s covid-19 laboratory based antibody test has been successfully evaluated by public health bodies for England, Scotland, and Wales as highly specific and they have communicated evaluated specificity results between 99.73% and 100%.”

The company said it has “commenced shipping to NHS laboratories” and has the “capacity to provide five million tests to the UK each month with immediate effect.”

While the results of the PHE’s evaluation remain unpublished, Abbott highlighted tests carried out by the University of Washington and published in the Journal of Clinical Microbiology. This claimed 99.9% specificity on 1020 patient samples and 100% sensitivity on 689 serum samples (from 125 people) when testing 17 days after symptoms began. One of the study authors reported personal fees from Abbott outside of the evaluation.1

Practising good science

But Jon Deeks, professor of biostatistics at the University of Birmingham, told The BMJ, “It’s not acceptable for data not to be fully reported and scrutinised for these tests.”

He said it is “very difficult to make any sense of the claims being made by the companies when the study report is not available. Part of practising good science is ensuring study results are reported and communicated appropriately, and can be shown to be supported by data. PHE needs to ensure this process is properly managed.”

Others noted that at least the study design or some details of the PHE evaluation could be made available.

Carl Heneghan, director of the centre for evidence based medicine at the University of Oxford, said, “In the current outbreak, the need for expediency has meant the normal rules for developing and publishing evidence have at times been lacking. We therefore need to be more vigilant about appraising and assessing the methods and results of emerging evidence.

“Because the study has not yet been published, we can’t tell whether the results are reliable or not, and who they apply to. This is obvious, but some of this uncertainty could be circumvented by publishing the protocol including information on the population, the test, and the referenced standard used.”

Sheila Bird, formerly programme leader at the MRC Biostatistics Unit at the University of Cambridge, also called for the study design to be published. She said, “Results should be reported publicly in a statistically appropriate manner. Details matter.”

Just the beginning

Arne Akbar, president of the British Society for Immunology, welcomed PHE’s approval but said this was just the “first step.”

“A good antibody test needs to be extremely accurate to make sure that it both correctly identifies all people who have these antibodies and that it doesn’t give false positives or incorrectly identify someone as having been exposed to covid-19. PHE will have carried out extensive examination on this test to look at these factors and it’s reassuring to see them give this the green light.

“However, developing a reliable antibody test is only the first step on the road to being able to understand how any immunity against covid-19 works and being able to use this knowledge to manage the spread of the disease.”

He said that while a positive result from a “reliable antibody test will tell us if that person has encountered the virus in the past and mounted an immune response,” it does not show if these antibodies will stop the person getting sick from covid-19 in the future or how long any protection generated might last.