Intended for healthcare professionals

Rapid response to:


Covid-19: FDA authorizes Moderna vaccine as US starts vaccinating health workers

BMJ 2020; 371 doi: (Published 21 December 2020) Cite this as: BMJ 2020;371:m4924

Read our latest coverage of the coronavirus outbreak

Rapid Response:

Blindness breach powered by inadvertent placebo proneness may confound COVID-19 vaccination study results.

Dear Editor,

COVID-19 vaccination studies are currently being conducted under time pressure. Thus, many subjects are enrolled in order to quickly get to a sufficient, a priori determined number of COVID-19 diagnosed subjects. We hypothesize that a possible confounding of reported efficacy could result from blindness breach (BB), in this case powered by the high number of included subjects compared to the few diagnosed cases, which may inadvertently result in biased diagnosis in placebo prone individuals.

Pragmatically, one can see each of these studies (Pfizer, Moderna, Oxford-AstraZeneca) as a study-within-study: A small ‘diagnosed-subjects study’ (subjects being COVID-19 diagnosed after being vaccinated or not) is incorporated within a large vaccination study (of all people enrolled). E.g. in the Oxford-AstraZeneca COVID-19 vaccination study with reported 90% efficacy, +/- 44 subjects were COVID-19 diagnosed within a vaccination study population of 2.74K. The question in this is how many of the 44 are from the placebo versus verum group.

In such a setting, the effect of BB may be powered by numbers, as follows. The molecular testing leads to an intersection of two sets (the set that is infected, whether showing symptoms or not, and the set that is showing symptoms, whether infected or not) which results in a diagnosis partially informed by subjective symptoms. The subjects within this intersection form the population of the ‘diagnosed-subjects study’ to see in retrospect who has been vaccinated. As in any suggestibility test, some people are more suggestible (not ‘gullible’!) than others. In the same vein, some people are more placebo-prone than others, with health-related consequences. Within a large vaccination study population, there are bound to be highly suggestible/placebo-prone subjects. [1] Suggestible subjects may be more susceptible to feeling and reporting more pronounced subjective symptoms if, for example, they think to be in the placebo group, especially when feeling ‘something’ that might be attributable to the start of the disease, and with heightened attention from being in a study. Thus, in case of relatively small BB, they may be naturally distilled to create a reporting bias skewing the ‘diagnosed-subjects study,’ that may potentially lead to a higher reported efficacy. Moreover, there is a growing insight in the mind’s impact on the immune system. [2] One meta-analysis showed “more than 300 empirical articles describing a relationship between psychological stress and parameters of the immune system in human participants.” [3]

There is evidence that success of blinding is often not maintained in placebo controlled trials. [4] In a systematic review of 63 randomized placebo controlled trials, according to Park et al., "about 73.2% (46 out of 63) studies turned out unblinded". They generally contend that “the optimistic assumption that blinding remains intact as intended throughout the trial proved to be untrue.” [5] Moreover, the more rigorously double-blind a study, the less a treatment’s efficacy according to that study tends to become. [6][7] In the mRNA studies, the vaccine-administrators were non-blind. The Oxford-AstraZeneca study was single-blind in toto. Unreported non-consciously experienced vaccine side effects may also be relevant. We see in all this the possibility of a structural ‘vaccine blinded-RCT artifact,’ especially of large non-double-blind vaccination studies with a small number of diagnosed subjects.

Inadvertently distilled placebo proneness might differentially confound the Oxford-AstraZeneca sub-studies with 90% versus 62% efficacy based on even a relatively small difference in BB. There are many differences between both studies; they are in fact 2 separate phase 3 studies with different dosing, different age and cultural profile, different continents. [8] The collective efficacy of 70% from the Oxford-AstraZeneca sub-studies markedly differs from that of both mRNA COVID-19 vaccination studies showing >90% efficacy. In contrast to the use of active placebo in the Oxford-AstraZeneca study (except as second dose in the Brazil sub-study), the mRNA studies compare verum to passive (fewer side effects) placebo. This is a setting in which the role of BB may be substantially higher than in active placebo studies, as has repeatedly been shown in medication studies. [9] Also, their numbers of enrolled subjects are higher. Thus, the ‘vaccine blinded-RCT artifact’ may play a bigger role in the mRNA studies.

To mitigate the untoward effect of BB in future vaccine RCT studies, we propose assessIing treatment assumptions (perhaps more appropriately called ‘allocation assumptions’), which consist of properly asking the subjects to intuitively guess (assume) which group they belong to. This way, we get a more accurate idea of BB and its effects in a timely and low-cost manner. If even while not strictly knowing, many subjects guess correctly, this would indicate a substantial (albeit non-conscious) BB. Inasmuch as this is the case, vaccination studies may show the result of mental factors apart from and above the vaccine. This may prove useful in the management of COVID-19. For instance, “Stress-induced immune dysregulation has been shown to be significant enough to result in health consequences, including reducing the immune response to vaccines” [10]

This is urgent, among other reasons because once vaccinations are widely available on the market, it will become more challenging to keep those subjects with a negative treatment assumption from procuring a vaccination outside of the study. Also, with vaccination may come an enhanced lack of adherence to other protective measures. At present, we have no idea how big the influence of inadvertently distilled placebo proneness may be in the present vaccination studies, and whether and to what extent they have caused a reporting bias.

Dr. Jean-Luc Mommaerts, M.D., M.A.I., - Ph.D. - Vrije Universiteit Brussel (Free University Brussels)
Prof. Dr. Anne-Mieke Vandamme, Ph.D. - - KU Leuven - Clinical and Epidemiological Virology - Rega Institute for Medical Research

[1] Mommaerts JL, Devroey D. The placebo effect: how the subconscious fits in. Perspect Biol Med. 2012 Winter;55(1):43-58. doi: 10.1353/pbm.2012.0005. PMID: 22643715.
[2] Brod S, Rattazzi L, Piras G, D’Acquisto F. ‘As above, so below’ examining the interplay between emotion and the immune system. Immunology. 2014;143(3):311-318. doi:10.1111/imm.12341
[3] Segerstrom SC, Miller GE. Psychological stress and the human immune system: a meta-analytic study of 30 years of inquiry. Psychol Bull. 2004;130(4):601–630.
[4] Fergusson D, Glass KC, Waring D, Shapiro S. Turning a blind eye: the success of blinding reported in a random sample of randomised, placebo controlled trials. BMJ 328(7437, 432 (2004).
[5] Park J, Bang H, Cañette I. Blinding in clinical trials, time to do it better. Complement Ther Med 16(3): 121-123 (2008).
[6] Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273(5), 408–412 (1995).
[7] Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP. Does the quality of randomized trials affect estimates of intervention efficacy reported in meta-analysis? Lancet 352(9128), 609–613 (1998).
[9] Moncrieff J, Wessely S, Hardy R. Antidepressants using active placebos. Cochrane Database Syst Rev. 2001;(2):CD003012. doi: 10.1002/14651858.CD003012. Update in: Cochrane Database Syst Rev. 2004;(1):CD003012. PMID: 11406060.
[10] Godbout JP, Glaser R. Stress-induced immune dysregulation: implications for wound healing, infectious disease and cancer. J Neuroimmune Pharmacol. 2006;1(4):421–427.

Competing interests: Anne-Mieke Vandamme has received consultancy fees from Gilead for their virology advisory board.

23 December 2020
Jean-Luc Mommaerts
Medical doctor, researcher
Anne-Mieke Vandamme (KULeuven)
Vrije Universiteit Brussel
Edegem, Belgium