Intended for healthcare professionals

CCBY Open access
Research

Effectiveness of BNT162b2 and mRNA-1273 covid-19 vaccines against symptomatic SARS-CoV-2 infection and severe covid-19 outcomes in Ontario, Canada: test negative design study

BMJ 2021; 374 doi: https://doi.org/10.1136/bmj.n1943 (Published 20 August 2021) Cite this as: BMJ 2021;374:n1943

Read our latest coverage of the coronavirus pandemic

Linked Research

Effectiveness of the CoronaVac vaccine in older adults in Brazil

Linked Editorial

Real world effectiveness of covid-19 vaccines

  1. Hannah Chung, associate research methodologist1,
  2. Siyi He, research analyst1,
  3. Sharifa Nasreen, postdoctoral fellow1,
  4. Maria E Sundaram, postdoctoral fellow12,
  5. Sarah A Buchan, scientist and assistant professor1234,
  6. Sarah E Wilson, medical epidemiologist and assistant professor1234,
  7. Branson Chen, senior health information analyst1,
  8. Andrew Calzavara, associate research methodologist1,
  9. Deshayne B Fell, scientist and associate professor156,
  10. Peter C Austin, senior scientist and professor17,
  11. Kumanan Wilson, senior scientist and professor589,
  12. Kevin L Schwartz, infection prevention and control physician and assistant professor123,
  13. Kevin A Brown, scientist and assistant professor123,
  14. Jonathan B Gubbay, medical microbiologist and associate professor310,
  15. Nicole E Basta, associate professor11,
  16. Salaheddin M Mahmud, director and professor12,
  17. Christiaan H Righolt, analytics manager12,
  18. Lawrence W Svenson, provincial health analytics officer and associate professor13141516,
  19. Shannon E MacDonald, associate professor1517,
  20. Naveed Z Janjua, senior scientist and clinical professor1819,
  21. Mina Tadrous, scientist and assistant professor12021,
  22. Jeffrey C Kwong, senior scientist and professor12342223
  23. on behalf of the Canadian Immunization Research Network (CIRN) Provincial Collaborative Network (PCN) Investigators
  1. 1ICES, Toronto, ON, Canada
  2. 2Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
  3. 3Public Health Ontario, ON, Canada
  4. 4Centre for Vaccine Preventable Diseases, University of Toronto, Toronto, ON, Canada
  5. 5School of Epidemiology and Public Health, University of Ottawa, ON, Canada
  6. 6Children’s Hospital of Eastern Ontario Research Institute, Ottawa, ON, Canada
  7. 7Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada
  8. 8Bruyère and Ottawa Hospital Research Institutes, Ottawa, ON, Canada
  9. 9Department of Medicine, University of Ottawa, Ottawa, ON, Canada
  10. 10Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada
  11. 11Department of Epidemiology, Biostatistics, and Occupational Health, School of Population and Global Health, McGill University, Montreal, QC, Canada
  12. 12Vaccine and Drug Evaluation Centre, Department of Community Health Sciences, University of Manitoba, Winnipeg, MB, Canada
  13. 13Analytics and Performance Reporting Branch, Alberta Health, Edmonton, AB, Canada
  14. 14Division of Preventive Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
  15. 15School of Public Health, University of Alberta, Edmonton, AB, Canada
  16. 16Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
  17. 17Faculty of Nursing, University of Alberta, Edmonton, AB, Canada
  18. 18British Columbia Centre for Disease Control, Vancouver, BC, Canada
  19. 19School of Population and Public Health, University of British Columbia, Vancouver, BC, Canada
  20. 20Women’s College Hospital, Toronto, ON, Canada
  21. 21Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada
  22. 22Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
  23. 23University Health Network, Toronto, ON, Canada
  1. Correspondence to: J Kwong jeff.kwong{at}utoronto.ca (or @DrJeffKwong on Twitter)
  • Accepted 2 August 2021

Abstract

Objective To estimate the effectiveness of mRNA covid-19 vaccines against symptomatic infection and severe outcomes (hospital admission or death).

Design Test negative design study.

Setting Ontario, Canada between 14 December 2020 and 19 April 2021.

Participants 324 033 community dwelling people aged ≥16 years who had symptoms of covid-19 and were tested for SARS-CoV-2.

Interventions BNT162b2 (Pfizer-BioNTech) or mRNA-1273 (Moderna) vaccine.

Main outcome measures Laboratory confirmed SARS-CoV-2 by reverse transcription polymerase chain reaction (RT-PCR) and hospital admissions and deaths associated with SARS-CoV-2 infection. Multivariable logistic regression was adjusted for personal and clinical characteristics associated with SARS-CoV-2 and vaccine receipt to estimate vaccine effectiveness against symptomatic infection and severe outcomes.

Results Of 324 033 people with symptoms, 53 270 (16.4%) were positive for SARS-CoV-2 and 21 272 (6.6%) received at least one dose of vaccine. Among participants who tested positive, 2479 (4.7%) were admitted to hospital or died. Vaccine effectiveness against symptomatic infection observed ≥14 days after one dose was 60% (95% confidence interval 57% to 64%), increasing from 48% (41% to 54%) at 14-20 days after one dose to 71% (63% to 78%) at 35-41 days. Vaccine effectiveness observed ≥7 days after two doses was 91% (89% to 93%). Vaccine effectiveness against hospital admission or death observed ≥14 days after one dose was 70% (60% to 77%), increasing from 62% (44% to 75%) at 14-20 days to 91% (73% to 97%) at ≥35 days, whereas vaccine effectiveness observed ≥7 days after two doses was 98% (88% to 100%). For adults aged ≥70 years, vaccine effectiveness estimates were observed to be lower for intervals shortly after one dose but were comparable to those for younger people for all intervals after 28 days. After two doses, high vaccine effectiveness was observed against variants with the E484K mutation.

Conclusions Two doses of mRNA covid-19 vaccines were observed to be highly effective against symptomatic infection and severe outcomes. Vaccine effectiveness of one dose was observed to be lower, particularly for older adults shortly after the first dose.

Introduction

Understanding how efficacy estimates from clinical trials of covid-19 vaccines translate into effectiveness estimates in the real world is crucial, given differences in populations, dosing intervals, and emerging variants.1 Because of constraints on the supply of covid-19 vaccines, Canada’s National Advisory Committee on Immunization recommended extending the interval between doses to a maximum of 16 weeks.2 With constraints on vaccine supply globally, determining the effectiveness of these vaccines after a single dose versus two doses is important for informing policy for many countries.1

We applied the test negative design to linked, population based health databases in Ontario, Canada (population 15 million) to evaluate vaccine effectiveness against symptomatic SARS-CoV-2 infection and severe outcomes (hospital admission or death) associated with SARS-CoV-2 infection for two mRNA vaccines: BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna).

Methods

Study population, setting, and design

We conducted a test negative design study among community dwelling Ontarians who had symptoms consistent with covid-19. The test negative design is comparable to a nested case-control design, with individuals who have symptoms and are tested for the presence of a pathogen of interest serving as the nesting cohort.134 All Ontarians aged ≥16 years, eligible for provincial health insurance, not living in long term care, and who were tested for SARS-CoV-2 between 14 December 2020 and 19 April 2021 were eligible for inclusion. We excluded those who tested positive for SARS-CoV-2 before 14 December 2020 and recipients of the ChAdOx1 (Oxford-AstraZeneca) vaccine. We restricted the analysis to individuals who had at least one relevant covid-19 symptom (based on self-report or observation, such as measured temperature), at the time of testing, which was collected on the SARS-CoV-2 test requisition form (see supplementary methods).

Data sources and definitions

Using unique encoded identifiers, we linked data from provincial SARS-CoV-2 laboratory testing, covid-19 vaccination, and health administrative datasets and analysed them at ICES (formerly the Institute for Clinical Evaluative Sciences).

Outcomes

Our first primary outcome was symptomatic SARS-CoV-2 infection, ascertained by real time reverse transcription polymerase chain reaction (RT-PCR) tests on respiratory specimens, including samples from the nasopharynx (most common), nose, throat, saliva, and turbinates.5 Using data from the Ontario Laboratories Information System, which captured 91.8% (n=258 207) of all provincially reported cases of laboratory confirmed covid-19 (n=281 261) during the study period, people who tested positive were considered as cases and those who tested negative were considered as controls. Since the dates for symptom onset were inconsistently reported in the Ontario Laboratories Information System, we used the date of specimen collection as the index date. For cases with multiple positive test results, we used the date of the first positive test result. For controls with multiple negative test results, we used the date of a randomly selected negative test result as the index date.

We obtained information on variants and mutations from the Public Health Case and Contact Management system, which contains information on the clinical course of cases and the results of screening tests for N501Y and E484K mutations and whole genome sequencing results that identify specific variant of concern lineages (alpha (B.1.1.7), beta (B.1.351), gamma (P.1)). All RT-PCR positive specimens with cycle threshold values ≤35 were tested for the N501Y mutation (starting 3 February 2021) and the E484K mutation (starting 22 March 2021).6 We considered samples with positive N501Y and negative E484K mutations as alpha variants, and samples with positive N501Y and E484K mutations as beta or gamma variants. We combined the latter two lineages for our analysis because there were small numbers of cases identified using whole genome sequencing.

Our second primary outcome was severe disease associated with symptomatic SARS-CoV-2 infection, defined as either hospital admission or death with a recent positive test result, using the earliest of the specimen collection date or the hospital admission or death date as the index date. We identified these outcomes using the Case and Contact Management system (for both hospital admissions and deaths), the Canadian Institute for Health Information’s Discharge Abstract Database (for hospital admissions), and the Ontario Registered Persons Database (for deaths). For hospital admissions identified using the Discharge Abstract Database, a positive test result must have occurred within 14 days before or three days after admission. For deaths identified using the Registered Persons Database, a positive test result must have occurred within 30 days before death or within seven days post mortem. We used the same control group as for the first primary outcome analysis (ie, individuals with symptoms who tested negative for SARS-CoV-2).

Vaccination against covid-19

BNT162b2 became available in Ontario on 14 December 2020 and mRNA-1273 on 28 December 2020.7 The initial vaccination phase prioritised high risk populations such as older adults living in communal settings, healthcare workers (including non-patient facing staff working in healthcare institutions), adults living in Indigenous communities, and adults aged ≥80 years.7 Ontario had initially followed the manufacturers’ recommended dosing schedules (ie, a 21 day interval for BNT162b2 and a 28 day interval for mRNA-1273), but because of disruptions to vaccine supply in late January 2021, the interval was extended to 35-42 days for everyone except older adults living in communal settings, and Indigenous people. In early March, Ontario adopted the National Advisory Committee on Immunization’s recommendation to delay administering the second dose by up to 16 weeks for most individuals.89 Eligibility expanded over time, taking into account both age (ie, graduated expansion by decreasing age) and other high risk populations, such as people with certain health conditions and their care givers, certain essential frontline workers, and those aged ≥18 years living or working in communities with a high incidence of covid-19 (ie, those disproportionately affected by covid-19 and where transmission was still high). Adherence to these eligibility criteria varied across regions. As of 19 April 2021, 28% of adults in Ontario had received at least one dose of a covid-19 vaccine.10 Comprehensive documentation of all covid-19 vaccination events in Ontario, including product, date of being administered, and dose number, is recorded in real time into COVaxON, a centralised covid-19 vaccine information system. We used the COVaxON file containing events up to 25 April 2021 for these analyses, which likely had records of all vaccinations delivered by 19 April 2021.

Covariates

We obtained age, sex, and postal code of residence as of 14 December 2020 from the Registered Persons Database. We obtained the number of RT-PCR tests for each participant during the three months before 14 December 2020 from the Ontario Laboratories Information System to use as a proxy for highly tested individuals at increased risk of exposure to SARS-CoV-2 infection (eg, healthcare workers and care givers of long term care residents, who must also undergo serial SARS-CoV-2 testing). To capture temporal changes in viral activity and regional vaccine roll-out, we grouped dates of testing into two week periods. We determined the presence of comorbidities that increase the risk of severe covid-19,11 identified from various databases using validated algorithms and commonly accepted diagnostic codes, which have been described elsewhere.12 Receipt of influenza vaccination (a proxy for health behaviours) was ascertained during the 2019/20 or 2020/21 influenza season, or both, using physician and pharmacist billing claims in the Ontario Health Insurance Plan and Ontario Drug Benefit databases, respectively. We determined the public health unit of residence using the postal code and Statistics Canada Postal Code Conversion File Plus (version 7B) and grouped the units into larger regions. From 2016 census data we obtained information at the ecological level of dissemination area on four important social determinants of health (median household income, proportion of the working population employed as non-health essential workers (those unable to work from home), average number of people in each dwelling, and proportion of the population who self-identified as a visible minority).13 Dissemination areas generally contain 400-700 people. Supplementary eTable 1 provides details of these covariates.

Statistical analysis

To compare characteristics between test positive cases and test negative controls and between vaccinated and unvaccinated individuals, we conducted descriptive analyses and calculated standardised differences.

We used multivariable logistic regression models to estimate the odds ratio, comparing the odds of vaccination between test positive cases and test negative controls (with unvaccinated people as reference group). We estimated unadjusted and adjusted odds ratios accounting for all listed covariates. These covariates were selected a priori based on their known associations with SARS-CoV-2 infection or severity and receipt of a covid-19 vaccine21114 and were assessed as potential confounders (supplementary eTable 2).15 Vaccine effectiveness was calculated using the formula: vaccine effectiveness=(1−odds ratio)×100%. We assumed those without information on exposures, outcomes, or covariates in ICES’ data holdings to not have the exposure, outcome, or covariate, and they were categorised as such within the analyses.

For the primary analysis, we estimated overall vaccine effectiveness (for both mRNA vaccines combined) for those who received only one dose by their index date and those who received two doses by their index date. We considered index dates within varying intervals after vaccination.

Vaccine effectiveness was also estimated ≥14 days after the first dose (among those who only received one dose) and ≥7 days after the second dose,16 stratified by vaccine product (BNT162b2 or mRNA-1273), age group (16-39, 40-69, and ≥70 years), sex, presence of any comorbidity, epidemic wave (index dates 14 December 2020 to 7 February 2021, representing wave 2 in Ontario; 8 February 2021 to 21 March 2021, representing the period between wave 2 and wave 3; and 22 March 2021 to 19 April 2021, representing wave 3), and variant (earlier variant versus alpha versus beta or gamma). We also estimated vaccine effectiveness by varying intervals after vaccination, stratified by age group.

We repeated these analyses for severe outcomes, with adjustments to the intervals after vaccination due to reduced sample sizes. For example, we evaluated vaccine effectiveness for the entire period (≥0 days) after receipt of the second dose.

Finally, to assess whether systematic differences between vaccinated and unvaccinated individuals were adequately controlled for in the main analyses, in a sensitivity analysis we assessed vaccine effectiveness against symptomatic infection and severe outcomes by varying intervals among only those who were vaccinated (treating people vaccinated 0-13 days before the test as reference group).17

All analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC). Tests were two sided, with P<0.05 considered as significant.

Patient and public involvement

Although study participants contributed in important ways to this research, it was not feasible to involve them in the design, conduct, reporting, or dissemination plans of our research. We did not involve members of the public in this research owing to resource and time constraints.

Results

Overall, 2 171 449 unique individuals were tested for SARS-CoV-2 from 14 December 2020 to 19 April 2021. After excluding those who had SARS-CoV-2 infection before the study period and those who had received the ChAdOx1 vaccine, 60.5% of those remaining did not have symptoms consistent with covid-19 or had no information on symptoms recorded in the Ontario Laboratories Information System, 24.4% were recorded as asymptomatic, and 15.1% had symptoms consistent with covid-19 recorded at the time of testing (fig 1). Grouped together, those with covid-19-like symptoms and those deemed asymptomatic had similar characteristics to the remaining individuals, except for covid-19 vaccine uptake, public health unit region, and number of previous SARS-CoV-2 tests (supplementary eTable 3).

Fig 1
Fig 1

Community dwelling people aged ≥16 years included in tested cohort between 14 December 2020 and 19 April 2021 in Ontario, Canada. *Recorded symptoms not consistent with covid-19 (eg, anxiety, cancer, falls). RT-PCR=reverse transcription polymerase chain reaction; ChAdOx1=Oxford-AstraZeneca vaccine

Of the 324 033 people with symptoms who were tested, 53 270 (16.4%) tested positive for SARS-CoV-2, 42 567 (79.9%) had information available on tests for variants, 21 272 (6.6%) had received at least one dose of mRNA vaccine, and 4894 (1.5%) had received two doses (table 1). Among test positive cases, 2479 (4.7%) had a severe outcome, of whom 2035 were admitted to hospital and 444 died. Most cases admitted to hospital tested positive before or on the admission date (1728, 84.9%) and nearly all tested positive before death. Test positive cases were more likely to be male, to reside in Peel region or Toronto, and to have had no SARS-CoV-2 tests during the three months before the vaccination programme, less likely to have received an influenza vaccine, and more likely to reside in neighbourhoods with lower income, more people in each dwelling, and greater proportions of essential workers and people from visible minorities (table 1). Vaccinated people were older, less likely to be male, and more likely to have had multiple SARS-CoV-2 tests during the three months before the vaccination programme, a comorbidity, and received an influenza vaccine. Compared with recipients of the mRNA-1273 vaccine, recipients of the BNT162b2 vaccine were younger, more likely to be female, and less likely to have a comorbidity (supplementary eTable 4). The distribution of these vaccine products also differed by public health unit regions. Most participants (77% for BNT162b2, 76% for mRNA-1273) had received only one dose by the index date. Supplementary eFigure 1 presents the distribution of vaccine product over the study period.

Table 1

Characteristics of participants with symptoms tested for SARS-CoV-2 between 14 December 2020 and 19 April 2021 in Ontario, Canada. Values are numbers (percentages) unless stated otherwise

View this table:

Against symptomatic infection, adjusted vaccine effectiveness observed ≥14 days after only one dose was 60% (95% confidence interval 57% to 64%). This increased from 48% (41% to 54%) at 14-20 days to a plateau of 71% (63% to 78%) at 35-41 days (fig 2, supplementary eTable 5). A 16% increase in risk of symptomatic infection was observed 7-13 days after one dose (vaccine effectiveness −16%, −26% to −6%), but no increase was observed 0-6 days after one dose. Vaccine effectiveness observed ≥7 days after two doses was 91% (89% to 93%). Against severe outcomes of hospital admission or death, vaccine effectiveness observed ≥14 days after one dose was 70% (60% to 77%), increasing from 62% (44% to 75%) at 14-20 days to 91% (73% to 97%) at ≥35 days, whereas vaccine effectiveness observed ≥7 days after two doses was 98% (88% to 100%) (fig 3, supplementary eTable 5).

Fig 2
Fig 2

Unadjusted and adjusted vaccine effectiveness estimates of covid-19 mRNA vaccines (BNT162b2, mRNA-1273) against laboratory confirmed symptomatic SARS-CoV-2 infection by various intervals, between 14 December 2020 and 19 April 2021 in Ontario, Canada. Models were adjusted for age, sex, public health unit region, biweekly period of test, number of SARS-CoV-2 tests in the three months before 14 December 2020, presence of any comorbidity increasing the risk of severe covid-19, receipt of influenza vaccination in current or previous influenza season, and fifths of neighbourhood level household income, number of people in each dwelling, proportion of people employed as non-health essential workers, and self-identified visible minority

Fig 3
Fig 3

Unadjusted and adjusted vaccine effectiveness estimates of covid-19 mRNA vaccines (BNT162b2, mRNA-1273) against severe outcomes (hospital admission or death) associated with laboratory confirmed symptomatic SARS-CoV-2 infection by various intervals, between 14 December 2020 and 19 April 2021 in Ontario, Canada. Models were adjusted for age, sex, public health unit region, biweekly period of test, number of SARS-CoV-2 tests in the three months before 14 December 2020, presence of any comorbidity increasing the risk of severe covid-19, receipt of influenza vaccination in current or previous influenza season, and fifths of neighbourhood level household income, number of people in each dwelling, proportion of people employed as non-health essential workers, and self-identified visible minority

In subgroup analyses of vaccine effectiveness against symptomatic infection, higher effectiveness was observed ≥14 days after only one dose of mRNA-1273 versus one dose of BNT162b2 (which was consistent across all age groups), for younger people versus adults aged ≥70 years, for those with no comorbidities versus those with comorbidities, and against the earlier variant and alpha variant versus beta or gamma variant (although 95% confidence intervals for vaccine effectiveness estimates for variants overlapped) (fig 4, supplementary eTable 6). Vaccine effectiveness estimates observed ≥7 days after two doses were, however, high (all ≥88%) and comparable across all subgroups, including against variants with the E484K mutation. Against severe outcomes, higher vaccine effectiveness was observed ≥14 days after one dose for those aged 16-39 years, but effectiveness estimates after two doses were mostly similar across subgroups (fig 4, supplementary eTable 7).

Fig 4
Fig 4

Adjusted vaccine effectiveness estimates ≥14 days after dose 1 (for those who received only one dose) and ≥0 days after dose 2 of covid-19 mRNA vaccines (BNT162b2, mRNA-1273) by various factors, including vaccine product, patient characteristics, epidemic wave, and SARS-CoV-2 lineage against laboratory confirmed symptomatic SARS-CoV-2 infection and severe outcomes (hospital admission or death) between 14 December 2020 and 19 April 2021 in Ontario, Canada. Models were adjusted for age, sex, public health unit region, biweekly period of test, number of SARS-CoV-2 tests in the three months before 14 December 2020, presence of any comorbidity increasing the risk of severe covid-19, receipt of influenza vaccination in current or previous influenza season, and fifths of neighbourhood level household income, number of people living in each dwelling, proportion of people employed as non-health essential workers, and self-identified visible minority (unless adjusted variable was used for stratification). *For vaccine effectiveness estimates against symptomatic SARS-CoV-2 infection, this interval was ≥7 days after dose 2. Against severe outcomes, vaccine effectiveness was evaluated for the entire period (≥0 days) after receipt of the second dose owing to the small number of outcomes. For subgroup analyses by characteristic and SARS-CoV-2 lineage, individuals vaccinated with either mRNA vaccine were included

Among adults aged ≥70 years, vaccine effectiveness against symptomatic infection after one dose was observed to be 64% (46% to 76%) at 28-34 days and 85% (38% to 97%) at 42-48 days, whereas comparable effectiveness estimates were achieved sooner after one dose for younger people (fig 5, supplementary eTable 8). Furthermore, vaccine effectiveness against severe outcomes was observed to be similar at ≥35 days after one dose (93%, 71% to 98%) as after two doses (97%, 86% to 99%).

Fig 5
Fig 5

Adjusted vaccine effectiveness estimates ≥14 days after dose 1 and ≥0 days after dose 2 of covid-19 mRNA vaccines (BNT162b2, mRNA-1273) against laboratory confirmed symptomatic SARS-CoV-2 infection and severe outcomes (hospital admission or death) by age group, between 14 December 2020 and 19 April 2021 in Ontario, Canada. Models were adjusted for age, sex, public health unit region, biweekly period of test, number of SARS-CoV-2 tests in the three months before 14 December 2020, presence of any comorbidity increasing the risk of severe covid-19, receipt of influenza vaccination in current or previous influenza season, and fifths of neighbourhood level household income, number of people living in each dwelling, proportion of people employed as non-health essential workers, and self-identified visible minority (unless adjusted variable was used for stratification). *For vaccine effectiveness estimates against symptomatic SARS-CoV-2 infection, this interval was ≥7 days after dose 2. Against severe outcomes, vaccine effectiveness was evaluated for the entire period (≥0 days) after receipt of the second dose owing to the small number of outcomes

In the sensitivity analysis restricted to vaccinated people with those vaccinated 0-13 days before the index date serving as the reference group, vaccine effectiveness estimates against symptomatic infection were observed to be similar to those of the main analyses (supplementary eTable 5). For hospital admissions and deaths, however, the estimates differed for earlier vaccination intervals.

Discussion

Using the test negative study design, which mitigates selection bias from differences in health seeking behaviour between vaccinated and unvaccinated people, we estimated high (>90%) vaccine effectiveness of mRNA vaccines BNT162b2 and mRNA-1273 against symptomatic SARS-CoV-2 infection with full vaccination (≥7 days after a second dose), and moderate (about 50-70%) vaccine effectiveness with partial vaccination (≥14 days after only one dose). Estimates for both full and partial vaccination were about 10 percentage points higher against hospital admission or death than against symptomatic infection. Vaccine effectiveness generally was observed to increase over time after one dose; however, we also observed a slightly increased risk of symptomatic infection on days 7-13 after one dose, compared with no vaccination. In subgroup analyses, we observed lower vaccine effectiveness against symptomatic infection in adults aged ≥70 years and those with comorbidities, but a higher effectiveness after one dose of mRNA-1273 than after one dose of BNT162b2. Vaccine effectiveness was, however, observed to be consistently high across subgroups for fully vaccinated people, and also for older adults after longer intervals from a first dose.

Comparison with other studies

Our findings for fully vaccinated people are comparable with efficacy estimates from clinical trials and other real world effectiveness estimates reported in a range of settings.161819202122232425262728293031 Existing evidence for effectiveness estimates of one dose of mRNA vaccines from observational studies is heterogeneous,1627303132 with estimates for symptomatic infection ranging from 57% (95% confidence interval 50% to 63%)16 to 72% (58% to 86%)31 and post hoc calculations from efficacy trials about 90%.3334 Similar heterogeneity has been found among one dose effectiveness estimates in older adults,253235 with estimates generally lower for older adults after one dose,1632 and increasing over time. In our analysis we observed an effectiveness against symptomatic infection of 63% (95% confidence interval 40% to 72%) ≥49 days after only one dose, in keeping with the findings of several other studies.1630 In addition, we found that one dose of mRNA-1273 was associated with significantly higher effectiveness against symptomatic infection than one dose of BNT162b2. Differences in characteristics between recipients of the two vaccines might explain this finding, but similar results were also found in another Canadian province.36 Findings were, however, inconsistent in other studies that compared vaccine effectiveness between products after one dose; some found a trend towards higher effectiveness against infection using mRNA-1273,3738 whereas others found no difference.3940 However, the populations in each of these studies were more homogeneous than ours (eg, adults aged ≤40 years, healthcare workers, veterans). Our analysis also reflects extant evidence that vaccine effectiveness against symptomatic infection16 and covid-19 associated hospital admissions1923 increases to high levels after a second dose, even in older adults. Lastly, our finding that two doses of mRNA vaccines was not associated with appreciable vaccine escape by lineage alpha or variants with the E484K mutation (beta and gamma) is notable.

In our study, we observed an increased risk of infection 7-13 days after vaccination. Other studies also found an increased risk of SARS-CoV-2 infection up to 14 days after one dose.25314142 This could be due to an increase in exposures to SARS-CoV-2 after vaccination. Individuals might assume that they are protected against infection immediately after vaccination and engage in higher risk behaviours before a sufficient immune response has developed. Indeed, about 20% of the US public believe that protection is conferred either immediately or 1-2 weeks after the first vaccine dose.43 Future studies should examine the potential role of behavioural changes after the first dose of covid-19 vaccines. This finding could also be due a higher baseline risk of infection among those who were initially prioritised to receive the vaccine, which might not have been adequately controlled for in our models. A vaccine effectiveness estimate of close to 0% at 0-6 days after one dose, however, provides a level of validation that we had accounted for the differences between vaccinated and unvaccinated people.

Limitations of this study

This study has some limitations. Firstly, our study sample was limited to those with symptoms of covid-19 recorded in the Ontario Laboratories Information System, which decreased our potential sample size considerably, from 2 171 449 people who were tested for SARS-CoV-2 to 324 033 who had relevant covid-19 symptoms recorded in the information repository. Not all laboratories in Ontario currently have the information technology infrastructure to submit information on symptoms (or documentation of asymptomatic testing) recorded on the SARS-CoV-2 laboratory requisition form into the Ontario Laboratories Information System. Thus, the generalisability of our findings to the broader population is uncertain and we could not estimate vaccine effectiveness against asymptomatic infection. However, the per cent positivity of our study sample (53 270/324 033=16.4%) did not differ much from that observed in Ontario during the study period (281 261/2 171 449=13.0%), and we would expect positivity to be higher for people with symptoms than those without symptoms. We also acknowledge that those with no information on symptoms recorded in the Ontario Laboratories Information System might have had symptoms at the time of testing, and those recorded as asymptomatic might have subsequently developed symptoms. In addition, covid-19 vaccination status might also be collected on the laboratory requisition form. This could bias the true vaccine effectiveness estimate, depending on whether symptoms were more likely to be documented on requisition forms for vaccinated people who ultimately test positive for SARS-CoV-2 (this would bias vaccine effectiveness towards the null) or less likely to be recorded (this would bias vaccine effectiveness away from the null). To minimise this selection bias, traditional test negative design studies collect vaccination status among all people with symptoms consistent with the pathogen under study. However, the congruence of our findings for fully vaccinated people with extant studies provides some reassurance that any underestimation or overestimation of vaccine effectiveness is likely to be small. Secondly, because the date of symptom onset is largely unavailable in the Ontario Laboratories Information System, and the Case and Contact Management system only has information on people who test positive, we used specimen collection date as the index date. This might have led to some vaccinated people being classified into an incorrect dose-to-index interval because symptom onset would have occurred several days before being tested. The impact of earlier intervals on vaccine effectiveness estimates if using the specimen collection date depends on the test results for vaccinated people (eg, vaccine effectiveness for earlier intervals would be overestimated if there were more vaccinated test positive cases and underestimated if there were more vaccinated test negative controls) or whether the lag between symptom onset and specimen collection dates resulted in misclassification of vaccinated people (false negatives), which would overestimate vaccine effectiveness for earlier intervals. Furthermore, we could not limit the study population to those tested within 10 days of symptom onset, a commonly used inclusion criterion for test negative studies. Prolonging the interval between symptom onset and testing increases the likelihood of false negative cases, which lowers vaccine effectiveness estimates. However, 89% of cases with dates for both symptom onset and specimen collection documented in the Case and Contact Management system (not the source of symptom data for this study) were tested within 10 days of symptom onset. Thirdly, our results might have been affected by outcome misclassification of severe outcomes due to unlinked case records and incomplete capture of severe outcomes in the Case and Contact Management system, and delays in identifying hospital admissions in the Discharge Abstract Database (which depends on people being discharged) and deaths in the Registered Persons Database. The direction of bias to vaccine effectiveness estimates depends on whether data completeness and lags are differential between vaccinated and unvaccinated test positive cases. For example, if vaccination status ascertained during the case management process influenced the degree of data collection (eg, if more complete among vaccinated cases), vaccine effectiveness would be biased towards the null, or if unvaccinated cases have a prolonged hospital admission because of more severe course of illness, their hospital record would not be available for analysis and vaccine effectiveness would be biased towards the null. Fourthly, some of our covariates might be subject to measurement error. We used frequency of previous SARS-CoV-2 tests as a proxy to identify those at higher risk of exposure (and increased likelihood to be targeted for early vaccination). However, we did not include point of care tests because they are incompletely captured in the Ontario Laboratories Information System. Furthermore, since access to testing is variable, we might not have adequately controlled for this situation. Finally, we might not have adequately accounted for confounding bias with the covariates that were available in the study databases, especially against hospital admissions and deaths.

Conclusions

Our findings suggest that older people and those with comorbidities might benefit from risk based recommendations to minimise delays before a second dose of mRNA covid-19 vaccine. However, increasing protection against severe outcomes—arguably the more important outcomes—with increasing time after a first dose provides support for delaying the second dose in settings with constraints on vaccine supply. Mathematical modelling could be done to show how, particularly for jurisdictions with limited vaccine supply, vaccines should be distributed to maximise the protection of populations (eg, the relative benefits of providing second doses earlier to older populations versus providing more first doses to younger populations who respond better to one dose and thereby leading to more rapid achievement of herd immunity by maximising coverage with one dose). Since vaccine effectiveness against symptomatic infection after one dose is only moderate, and among older adults appears to be modest even at 14-20 days, people need to be informed that besides the absence of benefit during the first two weeks (and likely longer for older adults) after one dose of a mRNA covid-19 vaccine, they should continue to adhere to recommended public health measures, such as wearing a mask, physical distancing, and avoidance of social gatherings.

What is already known on this topic

  • Clinical trials and real world effectiveness studies have shown that mRNA covid-19 vaccines are protective against symptomatic SARS-CoV-2 infections and associated severe outcomes such as hospital admission and death

  • Greater protection is conferred after two doses administered at least three weeks apart

What this study adds

  • Protection against symptomatic infection and severe outcomes increases gradually after one dose of an mRNA covid-19 vaccine

  • Although lower vaccine effectiveness was observed in older adults shortly after one dose, levels of protection similar to those of younger people were observed after longer intervals

  • These findings support delaying second doses when vaccine supply is limited

Ethics statements

Ethical approval

Projects that use data collected by ICES under section 45 of Ontario’s Personal Health Information Protection Act (PHIPA), and use no other data, are exempt from research ethics board review. The use of the data in this project is authorised under section 45 and approved by ICES’ Privacy and Legal Office. ICES is a prescribed entity under PHIPA. Section 45 of PHIPA authorises ICES to collect personal health information, without consent, for the purpose of analysis or compiling statistical information with respect to the management of, evaluation or monitoring of, the allocation of resources to or planning for all or part of the health system.

Data availability statement

The study dataset is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (eg, healthcare organisations and government) prohibit ICES from making the dataset publicly available, access might be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS (email das@ices.on.ca). The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs might rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or require modification.

Acknowledgments

We thank Public Health Ontario for access to case level data from the Public Health Case and Contact Management system and covid-19 laboratory data, as well as assistance with data interpretation; the staff of Ontario’s public health units who are responsible for covid-19 case and contact management and data collection within the Case and Contact Management system; IQVIA Solutions Canada for use of its Drug Information Database; and the Ontario residents without whom this research would be impossible.

Footnotes

  • Contributors: HC and JCK designed and oversaw the study. SH and HC obtained the data and conducted all analyses (dataset and variable creation and statistical modelling). BC contributed to data analyses and data preparation for the symptomatic dataset. SN, MES, HC, and JCK drafted the manuscript. All authors contributed to the analysis plan, interpreted the results, critically reviewed and edited the manuscript, approved the final version, and agreed to be accountable for all aspects of the work. JCK is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: This work was supported by the Canadian Immunization Research Network (CIRN) through a grant from the Public Health Agency of Canada and the Canadian Institutes of Health Research (CNF 151944); funding from the Public Health Agency of Canada, through the Vaccine Surveillance Reference group and the COVID-19 Immunity Task Force; and ICES, which is funded by an annual grant from Ontario’s Ministry of Health (MOH) and Ministry of Long term Care (MLTC); and the Ontario Health Data Platform (OHDP), a Province of Ontario initiative to support Ontario’s ongoing response to covid-19 and its related impacts. JCK is supported by a clinician-scientist award from the University of Toronto Department of Family and Community Medicine. PCA is supported by a mid-career investigator award from the Heart and Stroke Foundation. The study sponsors did not participate in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

  • Parts of this material are based on data and/or information compiled and provided by the Canadian Institute for Health Information (CIHI) and by Cancer Care Ontario (CCO). However, the analyses, conclusions, opinions, and statement expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement by ICES, MOH, MLTC, OHDP, its partners, the Province of Ontario, CIHI, or CCO is intended or should be inferred.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Public Health Agency of Canada, the Canadian Institutes of Health Research, and Ontario’s Ministry of Health and Ministry of Long term Care for the submitted work. KW is chief executive officer of CANImmunize and serves on the data safety board for the Medicago covid-19 vaccine trial. SMM has received unrestricted research grants from Merck, GlaxoSmithKline, Sanofi Pasteur, Pfizer, and Roche-Assurex for unrelated studies. SMM has received fees as an advisory board member for GlaxoSmithKline, Merck, Pfizer, Sanofi Pasteur, and Seqirus. CHR has received an unrestricted research grant from Pfizer for an unrelated study.

  • The corresponding author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

  • Dissemination to participants and related patient and public communities: As the personal identifying information of participants have been removed from the study dataset, it is not possible to send the results of this study to participants. However, the results from this manuscript have been made publicly available through a preprint server and will be further disseminated by ICES through social media channels and news media.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

http://creativecommons.org/licenses/by/4.0/

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

References