Methodologic Considerations in COVID-19 Vaccine Trials
Peter Doshi’s commentary underscores the importance of understanding the type of outcomes the present COVID-19 vaccine studies can measure (1). From his investigation it is evident that these studies are not designed to address hard end points including number of hospitalizations, deaths or a reduction of transmission. Although these limitations are important to highlight, other potential methodologic limitations in these trials are worth mentioning.
Some of the published phase-2 studies of the SARS-CoV-2 vaccine have shown a 2 to 3.5 fold higher risk of adverse reactions (2,3) in the vaccine group compared with control subjects. Although in many of these studies the participants are blinded to treatment, it will be difficult to maintain blinding if more subjects in the vaccine group experience adverse events compared to the control subjects. A potentially higher adverse events risk in the vaccine group can lead to bias. For example, patients who suspect they may have received the vaccine (as a result of experiencing side effects) might feel more protected against the virus and engage in situations that might increase their chance of exposure to the virus. Conversely, patients in the control group who might be less likely to experience side effects might suspect not receiving the vaccine and choose to change their life style such that they are less likely to be exposed to the virus. This can potentially confound a true assessment of the benefits of the vaccine. This situation can also lead to selection bias since in some of the studies (Moderna trial) subjects were followed for up to two years and were allowed to have pre-existing conditions (4). A potentially higher rate of adverse events among the vaccinated group (compared with the control group) can lead to a subject’s withdrawal from the study due to a pre-existing condition. For example, joint pain secondary to the vaccine in a patient with pre-existing musculoskeletal disease (5) might prompt that subject to withdraw from the study leading to selection bias.
Although the COVID-19 vaccine studies have strong methodologic attributes including randomization, a potentially higher incidence of adverse events in the vaccine group along with a long follow up periods might make these studies prone to biases that are often found in observational studies.
1. Doshi P. Will covid-19 vaccines save lives? Current trials aren’t designed to tell us. BMJ 2020; 371 doi: https://doi.org/10.1136/bmj.m4037
2. Xia S, Duan K, Zhang Y, et al. Effect of an Inactivated Vaccine Against SARS-CoV-2 on Safety and Immunogenicity Outcomes: Interim Analysis of 2 Randomized Clinical Trials. JAMA. 2020;324(10):951-960.
3. Edward E. Walsh, Robert Frenck, Ann R. et al. RNA-Based COVID-19 Vaccine BNT162b2 Selected for a Pivotal Efficacy Study. medRxiv 2020.08.17.20176651; doi: https://doi.org/10.1101/2020.08.17.20176651
4. A Study to Evaluate Efficacy, Safety, and Immunogenicity of mRNA-1273 Vaccine in Adults Aged 18 Years and Older to Prevent COVID-19https://clinicaltrials.gov/ct2/show/NCT04470427
5. Folegatti PM, Ewer KJ, Aley PK et al. Oxford COVID Vaccine Trial Group. Safety and immunogenicity of the ChAdOx1 nCoV-19 vaccine against SARS-CoV-2: a preliminary report of a phase 1/2, single-blind, randomised controlled trial. Lancet. 2020 Aug 15;396(10249):467-478. doi: 10.1016/S0140-6736(20)31604-4
Competing interests: No competing interests