The imprinting effect of covid-19 vaccines: an expected selection bias in observational studies
BMJ 2023; 381 doi: https://doi.org/10.1136/bmj-2022-074404 (Published 07 June 2023) Cite this as: BMJ 2023;381:e074404Linked Fast Facts
Selection bias due to conditioning on a collider
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor
A striking phenomenon regarding COVID-19 vaccines, referred to as ‘immune imprinting’ or the more specific ‘negative effectiveness’, has been recently discussed here in The BMJ.1 Referring to Chemaitelly et al., which indicated that those with 3 doses of vaccine were more likely to be infected than those with 2,2 Monge et al. hypothesise that “the increased risk of reinfection in individuals vaccinated with a booster compared with no booster is the result” of a selection bias wherein those receiving the booster are those “more susceptible to reinfection”; a sort of counter to the hypothesised ‘healthy vaccinee bias’. Apart from the article’s inconclusive conclusion that this phenomenon “may be fully explained by selection bias”, this hypothesis would not apply to all such studies.
For example, while it could be reasonable to suppose that people opting for dose 3 and beyond would tend to be at higher risk of COVID-19, and thus more prone to reinfection, it is not obvious that this would apply to the recent study on healthcare workers presented by Shrestha et al.3 This study reveals an even greater problem. The phenomenon is not limited to boosters but is also found when comparing those receiving 2 doses to those receiving 0. In fact, Shrestha et al. indicates that each dose up to 3+ resulted in increased infections. And there are many other studies showing this phenomenon, also with regards to hospitalisations and deaths, in addition to the now widely accepted rapid waning of effectiveness, when comparing the double dosed to the unvaccinated, including another study with Chemaitelly as lead author.4 5 Several recently published papers also explain how counting window issues likely led to exaggerated effectiveness and safety estimates in both observational studies and clinical trials.6 7 8
The explanation offered up by Monge et al. fails. What we need is a proper explanation for perceived COVID-19 vaccine negative effectiveness, by the vaccine manufacturers or drug regulators. We need to know if this has always been the case or only since omicron, if the effect is dose-dependent, if certain groups are more at risk, etc. Otherwise, the notion that the benefits of the COVID-19 vaccines outweighs the risks is under threat. If the vaccines truly are negatively effective, it appears that the benefits do not outweigh the risks; there would be no benefits, and we simply add risks upon risks.
References
1. Monge S, Pastor-Barriuso R, Hernán MA. The imprinting effect of covid-19 vaccines: an expected selection bias in observational studies. BMJ. 2023;381:e074404. https://doi.org/10.1136/bmj-2022-074404.
2. Chemaitelly H, Ayoub HH, Tang P, et al. Long-term COVID-19 booster effectiveness by infection history and clinical vulnerability and immune imprinting: a retrospective population-based cohort study. The Lancet Infectious Diseases. 2023;23:816-27. https://doi.org/10.1016/S1473-3099(23)00058-0.
3. Shrestha NK, Burke PC, Nowacki AS, et al. Effectiveness of the Coronavirus Disease 2019 Bivalent Vaccine. Open Forum Infectious Diseases. 2023;10:ofad209. https://doi.org/10.1093/ofid/ofad209.
4. Goldberg Y, Mandel M, Bar-On YM, et al. Protection and Waning of Natural and Hybrid Immunity to SARS-CoV-2. New England Journal of Medicine. 2022;386:2201-12. https://www.nejm.org/doi/full/10.1056/NEJMoa2118946.
5. Chemaitelly H, Ayoub H, AlMukdad S, et al. Protection from previous natural infection compared with mRNA vaccination against SARS-CoV-2 infection and severe COVID-19 in Qatar: a retrospective cohort study. The Lancet Microbe. 2022;3:e944-55. https://doi.org/10.1016/S2666-5247(22)00287-7.
6. Fung K, Jones M, Doshi P. Sources of bias in observational studies of covid-19 vaccine effectiveness. Journal of Evaluation in Clinical Practice. 2023;1-7. https://doi.org/10.1111/jep.13839.
7. Lataster R. Reply to Fung et al. on COVID-19 vaccine case-counting window biases overstating vaccine effectiveness. Journal of Evaluation in Clinical Practice. 2023;1-4. https://doi.org/10.1111/jep.13892.
8. Doshi P, Fung K. How the case counting window affected vaccine efficacy calculations in randomized trials of COVID-19 vaccines. Journal of Evaluation in Clinical Practice. 2023;1-2. https://doi.org/10.1111/jep.13900.
Competing interests: No competing interests
Re: The imprinting effect of covid-19 vaccines: an expected selection bias in observational studies
Dear Editor
Monge and colleagues published a critique paper[1] of our study.[2] They argue that our observed immune imprinting effect reflects selection bias resulting from conditioning on a collider.[3] The cohorts in our study had different immune histories prior to being infected with Omicron, with one history being more protective than the other.[2] Consequently, conditioning on infection selects individuals with a higher infection propensity in the cohort that had the more protective immune history (the booster cohort).
While we appreciate their analysis and recognize the significance of investigating such potential bias, we disagree with how our study was portrayed and discussed.
First, if the proposed bias was present, its effect needs to be consistent throughout the follow-up period, rather than being limited to specific duration. Our findings, along with findings of our other investigations on imprinting, whether related to vaccination[2 4] or natural immunity,[5] indicate a time-dependent effect. The observed differences between cohorts emerged when the incidence was driven by Omicron BA.4/BA.5. These findings align with a biologically driven imprinting effect rather than a bias effect.
Second, our study employed rigorous exact matching to balance infection risk across the cohorts. This matching aimed to equalize the propensity to infection. While we acknowledge that no matching method can guarantee complete balance due to potential unobservable factors, the extensive matching was based on knowledge of SARS-CoV-2 epidemiology within this specific population, drawing from the collective insights of tens of COVID-19 studies.
Third, our study adjusted for testing frequency, an important contributor to variations in the "propensity" for documented infection. The relevance of our approach to control bias can be investigated by extending the simulations conducted by Monge and colleagues to examine how the effect size of bias changes, both in magnitude and direction, with the addition of each matching factor associated with the risk of infection in a simulated matched cohort study.
Fourth, in the presented simulations the observed bias effect sizes ranged from a risk ratio of 1.04-1.37. The maximum effect size of this bias (1.37) is smaller than our observed effect size of a hazard ratio of 1.47. This observation is noteworthy, considering that the simulation made a strong assumption about the differential in propensity for infection, an eightfold increase per each standard deviation in propensity. The simulation also did not incorporate matching or adjustment which conceivably should minimize the bias effect. Despite a strong assumption and not replicating our study matching and adjustment, the simulation did not generate an effect size as large as the one we observed.
Fifth, the critiqued analysis is part of a series of analyses that investigated imprinting effects.[2 4-6] In two of these analyses, which compared history of primary-series vaccination to no vaccination [2] and history of pre-Omicron infection to no pre-Omicron infection,[5] we observed a positive (not negative) imprinting effect, indicating an effect in the opposite direction of the purported bias effect. If collider bias existed, it would substantially underestimate the already strong positive imprinting effect, which is implausible given the large magnitude of the observed effect in the opposite direction. Moreover, the effect size was consistent for both analyses, despite differences in the immune histories, further supporting the notion that imprinting explains the observed results.
Sixth, we presented two additional analyses in the same study: one comparing history of primary-series vaccination to no vaccination and another comparing history of booster vaccination to no vaccination.[2] It is possible to predict the result of any of these three analyses from the other two by using a simple mathematical formula to combine effects. The result of the critiqued analysis can be predicted without conducting the analysis itself, by utilizing the results of the other two analyses. By employing this approach, the predicted result aligned with the actual result obtained in the critiqued analysis.
Seventh, subsequent to the critiqued study, we observed the same negative imprinting effect in another study that utilized a design not affected by the possibility of a collider bias.[4] An increasing number of studies, employing different designs and involving different populations, have reported similar effects.[6-9] Multiple basic science studies have reported imprinting effects in controlled laboratory experiments.[10-15] These effects have been observed for decades in relation to other infections, such as influenza.[16 17]
It is unfortunate that Monge and colleagues discussed our study in a context and language that suggested we are questioning the value of booster vaccination. Our studies consistently found that boosters are effective and represent an essential public health measure.[4 6 18-21] The imprinting effect was observed within a specific timeframe, after the complete waning of booster effectiveness, and coincided with infections involving new immune-evasive subvariants, namely BA.4/BA.5. [2 4 6] The effect arose from a specific mismatch between immune memory and actual immune challenge, as expected for an imprinting effect.[22] We highlighted that the identified effect may augment the need for repeated booster vaccination to blunt its effect.[2 4]
The title of Monge and colleagues' article and statements throughout the text make unwarranted and sweeping conclusions such as "an effect that cannot be correctly estimated with observational data". Such statements hinder progress in understanding COVID-19 immunity. We appreciate their critique, but respectfully disagree with their perspective on our study. Further investigation of imprinting has the potential to bring breakthroughs in our understanding of the mechanisms of COVID-19 immunity.
In conclusion, no observational study using real-world data can claim immunity against bias. Collider bias is one form of bias among others that can impact observational studies. The response to potential bias lies in designing studies to minimize it. A bias can exist, but its effect can be marginal or minimized by design. For a bias to exist it needs to coherently explain available evidence and its effect needs to be consistent in time and in immune-history design. We designed our study with careful consideration to mitigate different sources of bias.[2] The experience of this pandemic has shown the profound impact of observational studies such as ours in generating frontline scientific discoveries and informing vaccination guidelines. This has helped prevent severe disease and death.
References
1. Monge S, Pastor-Barriuso R, Hernan MA. The imprinting effect of covid-19 vaccines: an expected selection bias in observational studies. BMJ 2023;381:e074404. doi: 10.1136/bmj-2022-074404 [published Online First: 2023/06/08]
2. Chemaitelly H, Ayoub HH, Tang P, et al. COVID-19 primary series and booster vaccination and immune imprinting. medRxiv In press at Science Advances 2022:2022.10.31.22281756. doi: 10.1101/2022.10.31.22281756
3. Hernan MA, Monge S. Selection bias due to conditioning on a collider. BMJ 2023;381:1135. doi: 10.1136/bmj.p1135 [published Online First: 2023/06/08]
4. Chemaitelly H, Ayoub HH, Tang P, et al. Long-term COVID-19 booster effectiveness by infection history and clinical vulnerability and immune imprinting: a retrospective population-based cohort study. Lancet Infect Dis 2023;23(7):816-27. doi: 10.1016/S1473-3099(23)00058-0 [published Online First: 2023/03/14]
5. Chemaitelly H, Ayoub HH, Tang P, et al. Immune Imprinting and Protection against Repeat Reinfection with SARS-CoV-2. N Engl J Med 2022 doi: 10.1056/NEJMc2211055 [published Online First: 2022/10/13]
6. Qassim SH, Chemaitelly H, Ayoub HH, et al. Population immunity of natural infection, primary-series vaccination, and booster vaccination in Qatar during the COVID-19 pandemic: An observational study. medRxiv In press at eClinicalMedicine 2023:2023.04.28.23289254. doi: 10.1101/2023.04.28.23289254
7. Shrestha NK, Burke PC, Nowacki AS, et al. Effectiveness of the Coronavirus Disease 2019 Bivalent Vaccine. Open Forum Infect Dis 2023;10(6):ofad209. doi: 10.1093/ofid/ofad209 [published Online First: 2023/06/05]
8. Eythorsson E, Runolfsdottir HL, Ingvarsson RF, et al. Rate of SARS-CoV-2 Reinfection During an Omicron Wave in Iceland. JAMA Netw Open 2022;5(8):e2225320. doi: 10.1001/jamanetworkopen.2022.25320 [published Online First: 2022/08/04]
9. Shrestha NK, Shrestha P, Burke PC, et al. Coronavirus Disease 2019 Vaccine Boosting in Previously Infected or Vaccinated Individuals. Clin Infect Dis 2022;75(12):2169-77. doi: 10.1093/cid/ciac327 [published Online First: 2022/04/28]
10. Reynolds CJ, Pade C, Gibbons JM, et al. Immune boosting by B.1.1.529 (Omicron) depends on previous SARS-CoV-2 exposure. Science 2022:eabq1841. doi: 10.1126/science.abq1841 [published Online First: 2022/06/15]
11. Röltgen K, Nielsen SCA, Silva O, et al. Immune imprinting, breadth of variant recognition, and germinal center response in human SARS-CoV-2 infection and vaccination. Cell 2022;185(6):1025-40.e14. doi: 10.1016/j.cell.2022.01.018 [published Online First: 2022/02/13]
12. Collier A-r, Miller J, Hachmann N, et al. Immunogenicity of the BA.5 Bivalent mRNA Vaccine Boosters. bioRxiv 2022:2022.10.24.513619. doi: 10.1101/2022.10.24.513619
13. Wang Q, Bowen A, Valdez R, et al. Antibody responses to Omicron BA.4/BA.5 bivalent mRNA vaccine booster shot. bioRxiv 2022:2022.10.22.513349. doi: 10.1101/2022.10.22.513349
14. Cao Y, Jian F, Wang J, et al. Imprinted SARS-CoV-2 humoral immunity induces convergent Omicron RBD evolution. Nature 2023;614(7948):521-29. doi: 10.1038/s41586-022-05644-7 [published Online First: 2022/12/20]
15. Addetia A, Piccoli L, Case JB, et al. Neutralization, effector function and immune imprinting of Omicron variants. Nature 2023 doi: 10.1038/s41586-023-06487-6 [published Online First: 2023/08/31]
16. Janjua NZ, Skowronski DM, Hottes TS, et al. Seasonal influenza vaccine and increased risk of pandemic A/H1N1-related illness: first detection of the association in British Columbia, Canada. Clin Infect Dis 2010;51(9):1017-27. doi: 10.1086/656586 [published Online First: 2010/10/05]
17. Skowronski DM, Sabaiduc S, Leir S, et al. Paradoxical clade- and age-specific vaccine effectiveness during the 2018/19 influenza A(H3N2) epidemic in Canada: potential imprint-regulated effect of vaccine (I-REV). Euro Surveill 2019;24(46) doi: 10.2807/1560-7917.ES.2019.24.46.1900585 [published Online First: 2019/11/28]
18. Abu-Raddad LJ, Chemaitelly H, Ayoub HH, et al. Effect of mRNA Vaccine Boosters against SARS-CoV-2 Omicron Infection in Qatar. N Engl J Med 2022;386(19):1804-16. doi: 10.1056/NEJMoa2200797 [published Online First: 2022/03/10]
19. Altarawneh HN, Chemaitelly H, Ayoub HH, et al. Effects of Previous Infection and Vaccination on Symptomatic Omicron Infections. N Engl J Med 2022;387(1):21-34. doi: 10.1056/NEJMoa2203965 [published Online First: 2022/06/16]
20. Chemaitelly H, Ayoub HH, AlMukdad S, et al. Duration of mRNA vaccine protection against SARS-CoV-2 Omicron BA.1 and BA.2 subvariants in Qatar. Nat Commun 2022;13(1):3082. doi: 10.1038/s41467-022-30895-3 [published Online First: 2022/06/03]
21. Altarawneh HN, Chemaitelly H, Ayoub HH, et al. Effects of previous infection, vaccination, and hybrid immunity against symptomatic Alpha, Beta, and Delta infections. medRxiv 2023:2023.04.21.23288917. doi: 10.1101/2023.04.21.23288917
22. Wheatley AK, Fox A, Tan HX, et al. Immune imprinting and SARS-CoV-2 vaccine design. Trends Immunol 2021;42(11):956-59. doi: 10.1016/j.it.2021.09.001 [published Online First: 2021/09/29]
Competing interests: No competing interests
Professor Laith J. Abu-Raddad and Dr. Hiam Chemaitelly
Infectious Disease Epidemiology Group, Weill Cornell Medicine - Qatar, Qatar Foundation - Education City, P.O. Box 24144, Doha, Qatar. Telephone: +(974) 4492-8321. Fax: +(974) 4492-8333. E-mail: lja2002@qatar-med.cornell.edu
Competing interests: No competing interests