Intended for healthcare professionals

Rapid response to:

News

Covid-19: FDA authorizes Moderna vaccine as US starts vaccinating health workers

BMJ 2020; 371 doi: https://doi.org/10.1136/bmj.m4924 (Published 21 December 2020) Cite this as: BMJ 2020;371:m4924

Read our latest coverage of the coronavirus outbreak

Rapid Response:

COVID Vaccination Studies: From Double-Blind to Hardly-Blind?

Dear Editor,

Having read with interest the article in BMJ Opinion by Peter Doshi [COVS4] and also the response by John Stone [COVS3], we fully agree that "post-injection side-effects between vaccine and placebo might have allowed for educated guessing." [COVS4] In view of our prior reasoning [COVS5], this may have led to overestimating the vaccines' real-world effectiveness in morbidity and even mortality. We are fully aware that even with substantially less effectiveness, people need to be motivated to get a COVID vaccination, and any doubt should be weighed severely. Yet, in our view, the best motivation includes an open discussion based on facts. Withholding this for the sake of 'motivation' would have dire consequences eventually. The following should be read in light of this remark.

Until now, the FDA provided an Emergency Use Authorization for two mRNA vaccines: the Pfizer-BioNTech COVID-19 Vaccine and the Moderna COVID-19 Vaccine. Looking into the data, we notice a huge prevalence of side-effects in the vaccine arm compared to the placebo arm in both studies. While this does not lead subjects to certain knowledge about their allocations to vaccine vs. placebo arms, it logically decreases blindness, as argued in [COVS6].

As expanded upon in another response [COVS5], we see an inadvertent placebo proneness due to the study design common to huge placebo-controlled vaccine studies (many thousands of subjects) whereby only a fraction of subjects (+/ 170) get diagnosed. The mechanism is a kind of unwanted (inadvertent) distillation process. This inadvertently distilled placebo proneness may interact with even a small blindness breach towards an unknown degree of overvaluation of vaccines' efficacies.

As noted, a breach in blindness may come from side-effects. Indeed, there were many vaccination side-effects in both studies. For instance, in the Pfizer-BioNTech study, local pain was reported in subjects > 55 yrs., in the vaccine versus placebo arms (in %s) after the first shot: 71/9; after the second shot: 66/8 [COVS1]. Thus, eight times more people in this vaccine group vs. placebo group felt local pain. As to systemic effects in subjects > 55 yrs., we also see substantial differences. For example, as to fatigue (in %s): 51/17; headache 39/14; antipyretic use 38/10. All reported side-effects together, cumulatively, easily exceed 300%. On top of these come non-reported and even non-consciously felt side-effects. Thus, there is ample occasion for blindness breach of the educated guess-type.

In the Moderna briefing document to the FDA, we see for instance any local reaction in subjects > 64 yrs., in the vaccine versus placebo arms (in %s) after the first shot: 75/15; after the second shot: 84/13 [COVS2].

Also relevant to possible blindness breach is that the administrators of vaccines in both studies were nonblinded. Non-conscious/subconceptual communication may thus have contributed substantially to the unblinding of subjects.

Given the large numbers of participants in the studies, we see yet another possible route for blindness breach: the humanly understandable urge of participants to contact each other, either directly or through social media, in which case no personal contact is needed. For instance, subjects who know that local pain is very (eight times more, in reality) common and who don't feel any pain themselves, nor any other side-effect, can quite confidently guess they are in the placebo group. What would the authors do as naïve non-medical subjects? Honestly, both authors would probably 'cheat' a bit, maybe thinking it wouldn't hurt the study.

Note that, concerning effectiveness, we should not look at the sizes of both total studies, but of the groups of subjects getting ill. Thus, concerning effectiveness, these are studies of only modest sizes. Many communications about this are confusing. The relevant effectiveness studies are modestly sized.

Conclusion
Instead of huge double-blind studies, we see modestly sized, hardly-blind ones and an inadvertently enhanced placebo proneness. We sincerely hope the results will be as positive as they are believed to be. A prospective cohort study (the population's vaccination itself) is being rolled out on millions of subjects at present. We urge investigators to look closely into this and relate the findings to our arguments. A smaller than expected effectiveness should lead to taking these seriously and possibly re-evaluating the whole project. In any case, only the truth can decently guide us out of the pandemic, even if this truth is not what we all wish for.

Dr. Jean-Luc Mommaerts, M.D., M.A.I., - Ph.D. Jean-Luc.Mommaerts@vub.be - Vrije Universiteit Brussel (Free University Brussels)
Prof. dr. Dirk Devroey, M.D., Ph.D., Dean of Medicine & Farmacy - dirk.devroey@vub.be - Vrije Universiteit Brussel (Free University Brussels)

Bibliography
[COVS1] FDA Briefing Document - Pfizer-BioNTech COVID-19 Vaccine https://www.fda.gov/media/144245/download
[COVS2] FDA Briefing Document - Moderna COVID-19 Vaccine https://www.fda.gov/media/144434/download
[COVS3] John Strone "Is it possible to conduct a double-blind placebo vaccine study in humans?", 28 December, https://www.bmj.com/content/371/bmj.m4924/rr-3
[COVS4] Peter Doshi, "Pfizer and Moderna’s “95% effective” vaccines—let’s be cautious and first see the full data", November 26, 2020, https://blogs.bmj.com/bmj/2020/11/26/peter-doshi-pfizer-and-modernas-95-...
[COVS5] Jean-Luc Mommaerts and Anne-Mieke Vandamme, "Blindness breach powered by inadvertent placebo proneness may confound COVID-19 vaccination study results", 23 December 2020, https://www.bmj.com/content/371/bmj.m4924/rr-0
[COVS6] Jean-Luc Mommaerts, Dirk Devroey, Anne-Mieke Vandamme, "The Importance of Testing for Blindness in COVID-19 Vaccination Trials", 02 January 2021, https://www.bmj.com/content/371/bmj.m4924/rr-5

Competing interests: No competing interests

04 January 2021
Jean-Luc Mommaerts
Medical doctor, researcher
Dirk Devroey (VUB)
Vrije Universiteit Brussel
Laarbeeklaan 103 - 1090 Brussels, Belgium