Nirmatrelvir and risk of hospital admission or death in adults with covid-19: emulation of a randomized target trial using electronic health records
BMJ 2023; 381 doi: https://doi.org/10.1136/bmj-2022-073312 (Published 11 April 2023) Cite this as: BMJ 2023;381:e073312- Yan Xie, senior clinical epidemiologist1 2 3,
- Benjamin Bowe, senior biostatistician1 2,
- Ziyad Al-Aly, director1 2 4 5 6
- 1Clinical Epidemiology Center, Research and Development Service, VA Saint Louis Health Care System, Saint Louis, Missouri, USA
- 2Veterans Research and Education Foundation of Saint Louis, Saint Louis, Missouri, USA
- 3Department of Epidemiology and Biostatistics, College for Public Health and Social Justice, Saint Louis University, Saint Louis, Missouri, USA
- 4Department of Medicine, Washington University School of Medicine, Saint Louis, Missouri, USA
- 5Nephrology Section, Medicine Service, VA Saint Louis Health Care System, Saint Louis, Missouri, USA
- 6Institute for Public Health, Washington University in Saint Louis, Saint Louis, Missouri, USA
- Correspondence to: Z Al-Aly zalaly{at}gmail.com (or @zalaly on Twitter)
- Accepted 7 March 2023
Abstract
Objective To estimate the effectiveness of nirmatrelvir, compared with no treatment, in reducing admission to hospital or death at 30 days in people infected with the SARS-CoV-2 virus and at risk of developing severe disease, according to vaccination status and history of previous SARS-CoV-2 infection.
Design Emulation of a randomized target trial with electronic health records.
Setting Healthcare databases of the US Department of Veterans Affairs
Participants 256 288 participants with a SARS-CoV-2 positive test result and at least one risk factor for developing severe covid-19 disease, between 3 January and 30 November 2022. 31 524 were treated with nirmatrelvir within five days of testing positive for SARS-CoV-2 and 224 764 received no treatment.
Main outcome measures The effectiveness of starting nirmatrelvir within five days of a positive SARS-CoV-2 test result versus no treatment in reducing the risk of admission to hospital or death at 30 days was estimated in those who were not vaccinated, in those who received one or two doses of vaccine, and those who received a vaccine booster and, separately, in participants with a primary SARS-CoV-2 infection or reinfection. The inverse probability weighting method was used to balance personal and health characteristics between the groups. Relative risk and absolute risk reduction were computed from cumulative incidence at 30 days, estimated by weighted Kaplan-Meier estimator.
Results Among people who were not vaccinated (n=76 763; 5338 nirmatrelvir and 71 425 no treatment), compared with no treatment, the relative risk of nirmatrelvir in reducing admission to hospital or death at 30 days was 0.60 (95% confidence interval 0.50 to 0.71); the absolute risk reduction was 1.83% (95% confidence interval 1.29% to 2.49%). The relative risk and absolute risk reduction, compared with no treatment, were 0.65 (0.57 to 0.74) and 1.27% (0.90% to 1.61%), respectively, in people who received one or two doses of vaccine (n=84 620; 7989 nirmatrelvir and 76 631 no treatment); 0.64 (0.58 to 0.71) and 1.05% (0.85% to 1.27%) in individuals who received a booster dose of vaccine (n=94 905; 18 197 nirmatrelvir and 76 708 no treatment); 0.61 (0.57 to 0.65) and 1.36% (1.19% to 1.53%) in participants with a primary SARS-CoV-2 infection (n=228 081; 26 350 nirmatrelvir and 201 731 no treatment); and 0.74 (0.63 to 0.87) and 0.79% (0.36% to 1.18%) in participants who were reinfected with the SARS-CoV-2 virus (n=28 207; 5174 nirmatrelvir and 23 033 no treatment). Nirmatrelvir was associated with a reduced risk of admission to hospital or death in those aged ≤65 years and > 65 years; in men and women; in black and white participants; in those with 1-2, 3-4, and ≥5 risk factors for progression to severe covid-19 illness; and in those infected during the omicron BA.1 or BA.2 predominant era, and the BA.5 predominant era.
Conclusions In people with SARS-CoV-2 infection who were at risk of developing severe disease, compared with no treatment, nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days in people who were not vaccinated, vaccinated, and had received a booster vaccine, and in those with a primary SARS-CoV-2 infection and reinfection.
Introduction
Nirmatrelvir, given in combination with the enhancer ritonavir, is an orally administered protease inhibitor antiviral agent that blocks viral replication by targeting the SARS-CoV-2 3-chymotrypsin-like cysteine protease enzyme (Mpro).1 Evidence from one randomized placebo controlled trial indicated that oral nirmatrelvir, in combination with ritonavir, was effective in reducing admission to hospital or death at 28 days in individuals who were at risk of progression to severe covid-19 illness.2 This trial was conducted in the era before the omicron variant of the SARS-CoV-2 virus, included only individuals who were not vaccinated, and excluded those with a previous history of SARS-CoV-2 infection. Subsequently, an emergency use authorization was granted by regulatory agencies in the US, UK, and Europe, allowing the use of nirmatrelvir within five days of the onset of symptoms in people with SARS-CoV-2 infection who were at risk of progression to severe covid-19 illness. Real world evidence studies from Israel, Hong Kong, and the UK suggested that nirmatrelvir was associated with a reduced risk of admission to hospital or death during the early days of the omicron surge.345 Omicron has since replaced previous variants, has spawned subvariants, many people recently infected have been vaccinated, and many people have been reinfected (ie, had a previous SARS-CoV-2 infection).678 The effectiveness of nirmatrelvir in reducing the risk of admission to hospital or death by vaccination status and by previous history of SARS-CoV-2 infection is not known.9
Here, we used the national healthcare databases of the US Department of Veterans Affairs to emulate five target trials of oral nirmatrelvir versus no treatment in people in the community with at least one risk factor for progression to severe covid-19 illness. Risk factors included: age >60 years, body mass index >25, current smoker, chronic lung disease, cardiovascular disease, kidney disease, immune dysfunction, hypertension, diabetes, and cancer. Our aim was to estimate the effectiveness of nirmatrelvir, versus no treatment, started during the first five days of a diagnosis of covid-19, in reducing admission to hospital or death at 30 days in five trial populations. The trial populations included people who were not vaccinated, people who were vaccinated, those who had received a booster dose of vaccine, and those with a primary SARS-CoV-2 infection or who had been reinfected with the SARS-CoV-2 virus.
Methods
Specification of the target trial
We followed the approach, outlined by Hernán and Robins,10 of emulating a target trial with electronic health records. We first specified the trial protocol that would evaluate the research question of the effectiveness of nirmatrelvir, versus no treatment, started within the first five days of SARS-CoV-2 infection, in reducing the risk of admission to hospital or death at 30 days in people at risk of developing severe disease. We aimed to look at this question in five trial populations including people who were not vaccinated, those who had received one or two doses of the covid-19 vaccine, those who had received a booster dose, and those with a primary SARS-CoV-2 infection or who had been reinfected with the SARS-CoV-2 virus.1011 Table S1 lists the components of the target trial protocols.
The target trials enrolled participants with SARS-CoV-2 infection between 3 January and 30 November 2022 and with at least one risk factor for progression to severe covid-19 disease: age >60 years, body mass index >25, current smoker, chronic lung disease, cardiovascular disease, kidney disease, immune dysfunction, hypertension, diabetes, and cancer. Exclusion criteria were admission to hospital for covid-19 or use of other covid-19 treatments, including monoclonal antibodies or other antiviral agents, on the date of a SARS-CoV-2 positive test result. Other exclusion criteria were liver disease, end stage kidney disease, estimated glomerular filtration rate <30 mL/min/1.73 m2, or use of drug treatments that precluded nirmatrelvir treatment. The intervention of the study was treatment with nirmatrelvir or no treatment within five days of a SARS-CoV-2 positive test result. Outcome was admission to hospital or death within 30 days of a positive test result. The specified pragmatic trials were then emulated with the Veterans Affairs healthcare databases following the components listed in table S1 and detailed below.
Data sources
The study was conducted with data from the US Department of Veterans Affairs Health Care System, which has the largest integrated healthcare system in the US. The Veterans Affairs operates 171 Veterans Affairs Medical Centers and 1112 outpatient sites, and provides comprehensive healthcare to US veterans. Data domains used in this study included outpatient encounters, inpatient encounters, inpatient and outpatient drug treatments, and laboratory test results from the Veterans Affairs Corporate Data Warehouse. We used data from the Veterans Affairs covid-19 Shared Data Resource and the Corporate Data Warehouse laboratory domain to capture data on covid-19 test results. The area deprivation index was used as a composite measure of the contextual disadvantage at participants’ residential locations.12
Cohort
Between 3 January and 30 November 2022, we identified 311 407 participants with SARS-CoV-2 infection where their first record of infection was considered as T0. We further selected those with at least one risk factor for developing severe covid-19 disease: age >60 years, body mass index >25, current smoker, chronic lung disease, cardiovascular disease, kidney disease, immune dysfunction, hypertension, diabetes, and cancer (n=302 633). After removing participants with liver disease, end stage kidney disease, estimated glomerular filtration rate <30 mL/min/1.73 m2, or who were receiving drug treatments that precluded them from nirmatrelvir treatment, 292 222 remained in the cohort. We also removed those who had received other covid-19 treatments at T0, resulting in 276 198 participants, and those in hospital on the date of testing positive, giving a final inclusion cohort of 256 288 participants. In the overall cohort of 256 288 participants, 31 524 received nirmatrelvir within five days of T0 and 224 764 did not receive nirmatrelvir within five days of T0. Of those 256 288 participants included in the final cohort, 76 763 were not vaccinated (5338 nirmatrelvir, 71 425 no treatment), 84 620 had received one or two doses of vaccine (7989 nirmatrelvir, 76 631 no treatment), and 94 905 had received a booster dose of vaccine (18 197 nirmatrelvir, 76 708 no treatment). When we defined trial populations based on history of infection, 228 081 had a primary SARS-CoV-2 infection (26 350 nirmatrelvir, 201 731 no treatment) and 28 207 had a reinfection (5174 nirmatrelvir, 23 033 no treatment). Participants were followed until 12 December 2022. Figure 1 shows a flowchart of the construction of the cohort.
Outcomes
The composite outcome of admission to hospital or death within 30 days of a positive test result was examined in this study. Hospital records were obtained from inpatient admission data, and death records were obtained from the Veterans Affairs vital status database.
Covariates
Covariates were selected based on previous knowledge and assessed within the three years of T0 unless otherwise specified.131415161718192021222324252627 Figure S1 shows the directed acyclic graph which guided the selection of covariates.28 Personal and health factors were selected, including age, self-reported race (white, black, and other), sex (from self-reported sex), area deprivation index, body mass index, and smoking status (current, former, and never). We also identified laboratory measurements and diseases that might influence assignment of treatment and outcomes, including estimated glomerular filtration rate, systolic and diastolic blood pressure, cancer, chronic lung disease, dementia, immune dysfunction (including history of organ transplantation, HIV, or conditions requiring >30 days of treatment with corticosteroids or immunosuppressants, including systemic lupus erythematosus and rheumatoid arthritis), diabetes, and hyperlipidemia. We also adjusted for drug treatments that could have drug-drug interactions with nirmatrelvir-ritonavir, requiring: temporary suspension of the concomitant drug; adjustment of dosing of the concomitant drug; and monitoring for adverse effects.
Patient characteristics also included covid-19 vaccination status, influenza vaccination status, previous history of SARS-CoV-2 infection, use of steroids, and use of healthcare facilities, including the number of outpatient visits and hospital admissions, number of blood panel tests, and number of outpatient drug treatments received within one year of study enrollment. The number of outpatient and inpatient encounters through Medicare was also selected. Because the dynamic features of the pandemic might be associated with the use of nirmatrelvir and the study outcome, we specified further covariates to deal with these potential confounding effects, including calendar week of the SARS-CoV-2 infection, hospital bed capacity, hospital bed occupancy, and number of SARS-CoV-2 tests administered at the participants’ healthcare facility in the week of SARS-CoV-2 infection. In our cohort, 8.01% of participants had missing values for estimated glomerular filtration rate, 8.61% for body mass index, and 3.88% for blood pressure. Missing values were imputed based on all covariates by using multivariate imputation with chained equations and the predictive mean matching method.29 Restricted cubic spline functions were used to model potential non-linear relations between continuous variables and assignment of treatment.
Statistical analysis
For each emulated trial (not vaccinated, one or two doses of vaccine, booster dose, primary SARS-CoV-2 infection, and reinfection), baseline characteristics before and after weighting were presented. We used absolute standardized differences to evaluate the balance of baseline characteristics between groups, and a value <0.1 was considered evidence of good balance. We also examined the association between a set of prespecified sociodemographic, health, health behavior, and temporal characteristics, and the odds of receiving a prescription for nirmatrelvir with multivariate logistic regression.
Our aim was to examine the effect of starting nirmatrelvir in populations with the same baseline characteristics as the overall nirmatrelvir population but with a different vaccination status and reinfection status. Within each population group that met the criteria for one of the five specified trials (not vaccinated, one or two doses of vaccinate, booster dose, primary SARS-CoV-2 infection, and reinfection), we generated a propensity score from logistic regression for the probability of belonging to the overall nirmatrelvir population. The propensity score was then used to construct the inverse probability weight and applied to each trial population, which resulted in each trial population having similar baseline characteristics compared with the overall population that received nirmatrelvir.
Within each weighted population group, we then applied the clone method to estimate the effect of starting nirmatrelvir within five days of a SARS-CoV-2 positive test result, considering the time difference between infection and the start of treatment.10303132 The clone method was performed in three steps: cloning, censoring, and weighting (fig 2). First, we assigned each participant to both treatment strategies (nirmatrelvir or no treatment), so that at T0 each participant was cloned, and one copy of the observation (clone) was assigned to the nirmatrelvir group and one copy to the no treatment group. Second, we conducted censoring to ensure that participants followed their assigned strategy after T0. Because the no treatment group had a strategy of not receiving any covid-19 treatment during the treatment initiation period, the copies (or clones) in the no treatment group were censored on the day they received nirmatrelvir or other outpatient covid-19 treatments within the first five days of T0. Similarly, the copies in the nirmatrelvir group were censored on the day they received other outpatient covid-19 treatments before receiving nirmatrelvir or at the end of the treatment initiation period (five days after T0) for those who had not received treatment by that time.
Third, we applied weighting to deal with the selection bias introduced by censoring; this approach constructs a hypothetical population in which the occurrence of the censoring would be random. Within each cloned group and on each day during the first five day treatment initiation period, logistic regression was built to predict the probability of not being censored within those remaining in the group in the previous day. The inverse of this probability was then stabilized by the observed proportion of those remaining in the group and served as the weight on that day. Cumulative weighting was constructed by multiplying the weight from T0 (T0 was the date of the positive test result and day 1 of the treatment initiation period) up to that day and was then used as the final weight for that day.3233 Across the five emulated trials, the inverse probability of censoring weights had a mean ranging from 0.98 to 1.12 and standard deviation ranging from 0.01 to 0.15 (table S2), resulting in covariates that were well balanced between the groups. As a result, truncation of the weights was not undertaken. The final weight at day 5 (the end of the treatment initiation period) was used as the weight for subsequent days.
The effect was estimated by a weighted non-parametric Kaplan-Meier estimator.34 Cumulative incidence at 30 days and differences between groups were estimated and reported as a percentage at 30 days. Effectiveness on a relative scale, reported as relative risk, was estimated as the cumulative incidence in the nirmatrelvir arm compared with the no treatment arm.
We also repeated all of the analyses in the five emulated trials with an alternative approach of constructing the initial inverse probability weighting toward participants who received nirmatrelvir within the emulated trial population, instead of our primary approach of weighting toward all participants who received nirmatrelvir. For example, in the primary analyses, the effect of nirmatrelvir in the unvaccinated trial was estimated as an average treatment effect of those who were not vaccinated but with the same other baseline characteristics as those who received nirmatrelvir in the overall cohort across vaccination status. In this alternative approach, the effect of nirmatrelvir in the unvaccinated trial was estimated as an average treatment effect of those who had the same baseline characteristics as participants who received nirmatrelvir and were not vaccinated. The purpose of these analyses was to provide an estimate of effectiveness within the population that received the treatment in the real world and take into account the influence of baseline factors associated with the emulated trial population (eg, participants who received a booster vaccine were generally older and might have had a higher burden of comorbid conditions than those who were not vaccinated).
We further emulated several more trials to examine the effect of nirmatrelvir in populations with different characteristics, including age (≤65 years and >65 years), sex (men and women), race (white and black), number of risk factors for progression to severe covid-19 illness (1-2, 3-4, and ≥5 risk factors), and period of infection (BA.1 or BA.2 predominant era between 3 January and 25 June 2022, and BA.5 predominant era between 26 June and 5 November 2023).35 All populations were weighted toward all participants who received nirmatrelvir (as described for the primary analyses). We then examined if these characteristics modified the association between nirmatrelvir and admission to hospital or death by comparison of relative risks between populations.
We conducted multiple sensitivity analyses for each of the five emulated trials to examine the robustness of the findings. First, to remove the influence of events that occurred shortly after the SARS-CoV-2 positive test result, we examined the effect in those who did not have an event within the five day treatment initiation period (compared with the primary approach which allowed people who had an event in the five day treatment initiation period to be included and censored at the time the event occurred). Second, to test the alternative treatment strategy of starting treatment within the first three days of a positive test result, we examined the effect when defining the start of the treatment period as three days, compared with the five day treatment initiation strategy used in the primary approach. Third, to examine the sensitivity of the study results to our primary imputation method, we used a multiple imputation approach with 10 imputations, compared with single imputation used in the primary approach. Fourth, to further examine the influence of imputation, we conducted analyses only in cohort participants with complete data. Fifth, because some participants might cluster within the same healthcare system and share the same healthcare system level characteristics, we constructed a propensity score based on an hierarchical model where participants were specified as clustered within a healthcare system, compared with a one level model in the primary approach.36 Sixth, as an alternative analytic methodology to the approach based on propensity score used in the primary analyses, we used the instrumental variable method, where the number of nirmatrelvir prescriptions for each SARS-CoV-2 positive test result in the hospital within seven days of a participant testing positive was set as an instrumental variable, and we then performed two stage residual inclusion analyses.37383940
We used the approach outlined by Lipsitch et al to specify and test the association between nirmatrelvir and negative outcome controls where no previous knowledge supports a causal association.41 Admission to hospital or death within three days of the start of treatment was selected as the negative outcome control.9 The application of negative outcome controls might reduce (but does not eliminate) concerns about possible biases, including cohort design, covariate selection, determining outcome, and other sources of latent biases.
As a positive control, we excluded participants with a history of any covid-19 vaccination or history of SARS-CoV-2 infection, and those admitted to hospital in the first two days, and examined the risk over 28 days of follow-up.42 The purpose of this analysis was to test whether our approach would reproduce the protective association between nirmatrelvir and the risk of admission to hospital or death that was seen in the EPIC-HR (Study of Oral PF-07321332/Ritonavir Compared With Placebo in Nonhospitalized High Risk Adults With covid-19) trial.2
In all analyses, confidence intervals were generated based on non-parametric bootstrap with 500 replicates. Risk on the relative scale with 95% confidence interval that did not cross one and risk on the absolute scale with 95% confidence interval that did not cross zero was considered significant. We used SAS Enterprise Guide, version 8.2 (SAS Institute), to perform the analyses.
Patient and public involvement
Although there was no direct patient and public involvement in developing the research question, study design, and implementation, our resolve to do our part in the fight against the covid-19 global pandemic shaped our research agenda to identify and look at major knowledge gaps of great public health importance (including evaluation of the real world effectiveness of nirmatrelvir, the topic of these analyses). Our journey into covid-19 research was also inspired, shaped, and warmly supported by patients and patient advocacy groups from all corners of the world (North America, Central America, South America, Europe, Australia and New Zealand, East Asia, Middle East, and other regions). Drafts of this manuscript were shared with members of the public to obtain feedback, which has been incorporated into the revisions.
Results
We identified 31 524 participants who received nirmatrelvir and 224 764 who did not receive nirmatrelvir within five days of SARS-CoV-2 infection. Tables S3-S9 show the baseline characteristics of the overall cohort and the five emulated trial populations (not vaccinated, one or two doses of vaccine, booster dose, primary SARS-CoV-2 infection, and reinfection) and by treatment status (nirmatrelvir or no treatment) before weighting.
Table 1 shows the characteristics of the five emulated trial populations after weighting. Tables S10-14 describe the characteristics by treatment (nirmatrelvir or no treatment) at the end of the treatment initiation period (five days after SARS-CoV-2 infection) after application of the clone method. Examination of standardized mean differences in personal and health characteristics of those who received nirmatrelvir or no treatment at each day during the treatment initiation period (days 1-5) in the five emulated trial populations suggested good balance (figs S2-6).
Among those who were eligible for treatment with nirmatrelvir (ie, had at least one risk factor for progression to severe covid-19 illness), age >65 years (compared with age <65), white race (compared with black race), vaccination for covid-19 (compared with no vaccination), vaccination for influenza (compared with no vaccination), and SARS-CoV-2 infection later in 2022 (compared with January 2022) were associated with a higher probability of receiving nirmatrelvir (table S15). An increase in the number of risk factors was not associated with higher odds of receiving nirmatrelvir (table S15).
In individuals who were not vaccinated (n=76 763; 5338 nirmatrelvir, 71 425 no treatment) and after balancing baseline characteristics, 2.78% (95% confidence interval 2.36% to 3.17%) of participants in the nirmatrelvir group and 4.65% (4.29% to 4.97%) of participants in the no treatment group were admitted to hospital or died at 30 days. Nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days, with a relative risk of 0.60 (0.50 to 0.71) and absolute risk reduction of 1.83% (1.29% to 2.49%) compared with no treatment (fig 3, fig 4, and table S16).
In people who received one or two doses of vaccine (n=84 620; 7989 nirmatrelvir, 76 631 no treatment), 2.39% (95% confidence interval 2.08% to 2.71%) of participants in the nirmatrelvir group and 3.64% (3.43% to 3.88%) of participants in the no treatment group were admitted to hospital or died at 30 days. Nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days, with a relative risk of 0.65 (0.57 to 0.74) and absolute risk reduction of 1.27% (0.90% to 1.61%) compared with no treatment (fig 3, fig 4, and table S16).
In those that received a booster dose of vaccine (n=94 905; 18 197 nirmatrelvir, 76 708 no treatment), 1.90% (95% confidence interval 1.72% to 2.09%) of participants in the nirmatrelvir group and 2.94% (2.80% to 3.10%) of participants in the no treatment group were admitted to hospital or died at 30 days. Nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days, with a relative risk of 0.64 (0.58 to 0.71) and absolute risk reduction of 1.05% (0.85% to 1.27%) compared with no treatment (fig 3, fig 4, and table S16).
Among participants with a primary SARS-CoV-2 infection (n=228 081; 26 350 nirmatrelvir, 201 731 no treatment), 2.14% (95% confidence interval 1.96% to 2.29%) of participants in the nirmatrelvir group and 3.49% (3.36% to 3.61%) of participants in the no treatment group were admitted to hospital or died at 30 days. Nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days, with a relative risk of 0.61 (0.57 to 0.65) and absolute risk reduction of 1.36% (1.19% to 1.53%) compared with no treatment (fig 3, fig 4, and table S16).
In those who were reinfected with the SARS-CoV-2 virus (n=28 207; 5174 nirmatrelvir, 23 033 no treatment), 2.20% (95% confidence interval 1.86% to 2.57%) of participants in the nirmatrelvir group and 2.99% (2.71% to 3.31%) of participants in the no treatment group were admitted to hospital or died at 30 days. Nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days, with a relative risk of 0.74 (0.63 to 0.87) and absolute risk reduction of 0.79% (0.36% to 1.18%) compared with no treatment (fig 3, fig 4, and table S16).
We used an alternative approach to estimate the treatment effect of nirmatrelvir in the observed participants who received nirmatrelvir within each trial (instead of the primary approach that estimated the effect of nirmatrelvir in a population with characteristics similar to all participants who received nirmatrelvir across all trials); in this analysis, nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days (table S17).
We then examined the association in populations with different characteristics. Nirmatrelvir was associated with a reduced risk of admission to hospital or death in those aged ≤65 years and > 65 years; in men and women; in black and white participants; in those with 1-2, 3-4, and ≥5 risk factors for progression to severe covid-19 illness; and in those infected in the omicron BA.1 or BA.2 predominant era, and in the BA.5 predominant era (fig 5 and table S18). We did not find that age group, sex, race, number of risk factors, or predominant variants at the time of infection modified the association between nirmatrelvir and the outcome of admission to hospital or death.
Sensitivity analyses
The effectiveness of nirmatrelvir in each emulated trial was examined in multiple sensitivity analyses. First, we examined effectiveness in those without any event in the first five days of a SARS-CoV-2 positive test result (the primary approach included participants who had events in the first five days of a SARS-CoV-2 positive test result). Second, we examined the relative risk of starting nirmatrelvir within three days of a SARS-CoV-2 positive test result, compared with the five day treatment initiation period in the primary approach. Third, to examine the sensitivity of the study results to our primary imputation method, we conducted multiple imputation with 10 imputations, compared with single imputation used in the primary approach. Fourth, to further examine the influence of imputation, we conducted analyses only in cohort participants with complete data. Fifth, because some participants might cluster within the same healthcare system and share the same healthcare system level characteristics, we constructed a propensity score based on an hierarchical model where participants were specified as clustered within a healthcare system, compared with a one level model in the primary approach. Sixth, as an alternative analytic methodology to the approach based on propensity score used in the primary analyses, we used the instrumental variable method, where the number of nirmatrelvir prescriptions for each SARS-CoV-2 positive test result in the hospital within seven days of a participant testing positive was set as an instrumental variable, and then performed two stage residual inclusion analyses. The results from the sensitivity analyses were consistent, in direction and magnitude, with those obtained with our primary approach that, compared with no treatment, nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days (table S19).
Negative and positive controls
We tested the association between nirmatrelvir and admission to hospital or death within three days of the start of treatment as a negative control outcome. The results showed no significant association between receiving nirmatrelvir and the negative outcome controls across the five emulated trials (table S20). As a positive control, we emulated the EPIC-HR trial by applying similar eligibility criteria to determine if our approach reproduced knowledge generated in a randomized clinical trial. Consistent with the EPIC-HR study, our results showed that nirmatrelvir was associated with a reduced risk of admission to hospital or death at 28 days (table S20).
Discussion
In this real world analysis of 256 288 people with SARS-CoV-2 infection and at risk of developing severe covid-19 illness, 31 524 people were treated with nirmatrelvir and 224 764 people received no treatment within five days of a positive SARS-CoV-2 test result. Compared with no treatment, nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days in people who were not vaccinated, in those who received one or two doses of vaccine, in those who received a vaccine booster, and in those with a primary SARS-CoV-2 infection or who were reinfected with the virus. The real world effectiveness of nirmatrelvir was evident in those aged ≤ 65 years and > 65 years; in men and women; in black and white participants; in those with 1-2, 3-4, and ≥5 risk factors for progression to severe covid-19 illness; and in those infected in the omicron BA.1 or BA.2 predominant era, and in the BA.5 predominant era. The results were robust to challenge in the multiple sensitivity analyses, and the application of negative and positive controls gave results consistent with a priori expectations. Our findings provide evidence of the real world effectiveness of nirmatrelvir in reducing the risk of admission to hospital or death in people who were not vaccinated, in those who were vaccinated or received a booster dose, and in people with primary infection and SARS-CoV-2 reinfection.
Findings in relation to other studies
Although recent in vitro evidence suggested that nirmatrelvir has therapeutic value against covid-19 variants of concern, including omicron,4344454647 human data are lacking. Real world effectiveness studies from Israel, Hong Kong, and the UK provided evidence of the effectiveness of nirmatrelvir during the early days of the omicron surge when BA.1 or BA.2 was the predominant subvariant.345 SARS-CoV-2 is mutating rapidly, and new variants and subvariants are replacing previous ones every few months. Whereas BA.1 was the predominant subvariant of omicron in the US at the beginning of our study on 3 January 2022, BA.2 was predominant between 20 March and 25 June 2022, and the BA.5 subvariant between 26 June and 5 November 2022 (our study ended on 30 November 2022, and follow-up ended on 12 December 2022).678 Our study adds to the evidence base and provides estimates of the real world effectiveness of nirmatrelvir in the broader omicron era (between 3 January and 30 November 2022) and in the eras predominated by BA.1 or BA.2, and BA.5.
Most people infected in the omicron predominant era had been vaccinated (80% of people in the US, 80% in the UK, and 69.1% in the world had received at least one dose of a covid-19 vaccine) and a growing number of people have been reinfected. Our study suggests that the effectiveness of nirmatrelvir is maintained in people who are not vaccinated, in those who are vaccinated, or have received a booster dose, and in those with a primary infection or reinfection.21
The EPIC-HR trial, conducted before the omicron era, reported an absolute risk reduction of 5.62% at 28 days; this finding corresponded to an 88% relative risk reduction in admission to hospital or death.2 Our results suggested an absolute risk reduction of 1.83%, 1.27%, and 1.05%, and relative risk reduction of 40%, 35%, and 36% at 30 days in the unvaccinated, vaccinated, and booster groups, respectively. Absolute risk reductions were 1.36% and 0.79% and relative risk reductions were 49% and 36% in people with a primary infection and reinfection, respectively. In the positive control analyses where we emulated (to the extent possible) the EPIC-HR trial, the absolute risk difference was 2.08% and the relative risk reduction was 47%. The more modest absolute and relative risk reductions in our report might be related to the properties of the virus (omicron in our study v delta in the EPIC-HR trial) and potential differences in adherence to treatment between a randomized trial setting and a real world setting. These observations emphasize the importance and value of testing the real world effectiveness of treatments in populations and settings where they are actually used.48 Although the risk reductions on the relative and absolute scales were more modest than those seen in the EPIC-HR trial, the risk reduction for admission to hospital or death after nirmatrelvir treatment in this high risk population was not trivial, suggesting that despite mutations of the virus, people who are vaccinated, have received a booster dose, and those who had a previous infection still get substantial benefit from treatment with nirmatrelvir.
Despite the ubiquitous availability of nirmatrelvir free of charge to all enrollees of the Veterans Affairs healthcare system (who are entitled to Veterans Affairs healthcare services), and although we restricted our study population to those at risk of progression to severe covid-19 illness and who were eligible to receive nirmatrelvir, 31 524 people (12%) received nirmatrelvir compared with 224 764 people (88%) who did not receive outpatient covid-19 treatment during the study period. Our analyses evaluating the characteristics associated with the use of nirmatrelvir among those who were eligible (ie, had at least one risk factor for progression to severe covid-19 illness) suggested that younger age, black race, and health behaviors (including those who did not receive covid-19 or influenza vaccines) were markers of underuse, and that use of nirmatrelvir improved over time (greater in late 2022 compared with January 2022). These findings reflect the broader national underuse of nirmatrelvir49 and also the disproportionate use across racial groups and in those people who might be reluctant to receive a vaccine. A greater understanding is needed of the barriers to improved use in all eligible individuals and specifically among some racial groups and those who are not vaccinated. An urgent need exists for development of strategies to reduce this disproportionate underuse.
The covid-19 pandemic has emphasized the need for fast, reliable, and actionable evidence to inform public health decision making.5051 The emergence of new SARS-CoV-2 variants and subvariants over a short period of time has highlighted the limitations of randomized trials in looking at knowledge gaps about the effectiveness of treatments against these emerging variants (or subvariants). By the time a randomized trial is designed and implemented, the variant or subvariant of interest could likely be replaced, and the results might not be available in time to inform decision making. Using real world observational data from high quality integrated electronic health databases that are updated in real or near real time, advances in causal inference methodologies, and a target trial emulation approach might give actionable results much faster and provide much needed evidence to inform public health policies.505152
Strengths and limitations of this study
This study had several strengths. The Veterans Affairs operates the largest integrated healthcare system in the US; the extensive national healthcare databases of the US Department of Veterans Affairs allowed us to examine a larger number of treated and untreated patients and events than published randomized trials. We used these rich data and methodologic advances in causal inference to design and emulate target trials to estimate the effectiveness of nirmatrelvir versus no treatment in reducing the risk of admission to hospital or death in people infected with SARS-CoV-2 in the broader omicron era and in the eras predominated by the BA.1 or BA.2, and BA.5 subvariants of omicron. To inform treatment decisions, we specified and emulated five trials in key populations, including by vaccination status and reinfection status. By inverse probability weighting, we adjusted for a comprehensive list of covariates, including personal characteristics, vital signs, comorbidities, laboratory test results, drug treatments, and contextual characteristics. We also investigated the effectiveness in population groups according to age, sex, race, number of risk factors, and predominant omicron variant era. We challenged the robustness of our results in multiple sensitivity analyses and the results were consistent with the primary analyses. We also successfully tested negative and positive controls.
This study had several limitations. Although we used validated variable definitions and adjusted for a range of predefined variables from various data domains, misclassification bias and residual confounding might still exist. The Veterans Affairs population comprises mostly white men, which might limit the generalizability of the study findings. Although we comprehensively captured Veterans Affairs prescription records, our data might have missed nirmatrelvir prescribed outside Veterans Affairs, which might have reduced the estimated effectiveness of nirmatrelvir. We integrated multiple Veterans Affairs data domains and complemented these with external data sources, but we could have missed admissions to hospitals outside Veterans Affairs that were not reported to Veterans Affairs; however, this effect is unlikely to be different across the balanced exposure groups. We accounted for a range of personal and health characteristics that might potentially confound the association and successfully showed balance of exposure groups and application of negative controls. If people with suboptimal (and unaccounted for) health characteristics opt not to receive nirmatrelvir, however, these people might be at a relatively higher risk of adverse outcomes and this bias might lead to overestimating the effectiveness of nirmatrelvir. Conversely, if people with more optimal (and unaccounted for) health characteristics opt not to receive treatment, these people might have relatively more favorable outcomes and this bias might lead to underestimating the effectiveness of nirmatrelvir. Finally, as the virus continues to mutate, new variants emerge, newer vaccines become available, and lower risk groups receive nirmatrelvir treatment, the real world effectiveness of nirmatrelvir might also change over time.
Conclusions
In this study, we found that in people with SARS-CoV-2 infection who were at risk of developing severe disease, compared with no treatment, starting nirmatrelvir within five days of a positive SARS-CoV-2 test result was associated with a reduced risk of admission to hospital or death at 30 days. We saw this effect in people who were not vaccinated, in those who were vaccinated and those that had received a booster dose, and in those with a primary SARS-CoV-2 infection and reinfection.
What is already known on this topic
Nirmatrelvir reduced the risk of admission to hospital or death in one placebo controlled randomized trial, conducted before omicron was the predominant variant, in people with SARS-CoV-2 infection who were not vaccinated and had no previous history of SARS-CoV-2 infection
What this study adds
During the omicron predominant era, in people with SARS-CoV-2 infection who were at risk of developing severe covid-19 disease, nirmatrelvir was associated with a reduced risk of admission to hospital or death at 30 days compared with no treatment
The reduced risk was seen in people who were not vaccinated, in those who were vaccinated, those who had received a booster vaccine dose, and in those with a primary SARS-CoV-2 infection or who have been reinfected with the virus
Ethics statements
Ethical approval
This research project was reviewed and approved by the institutional review board of the Veterans Affairs Saint Louis Health Care System (protocol number 1606333) which granted a waiver of informed consent.
Data availability statement
The data that support the findings of this study are available from the US Department of Veterans Affairs. Veterans Affairs data are made freely available to researchers behind the Veterans Affairs firewall with an approved Veterans Affairs study protocol. For more information, please visit www.virec.research.va.gov or contact the Veterans Affairs Information Resource Center (VIReC) at VIReC@va.gov
Acknowledgments
This study used data from the Veterans Affairs covid-19 Shared Data Resource.
Footnotes
Contributors: ZA-A and YX contributed to the development of the study concept and design, and to the data analysis and interpretation of the results, and drafted the manuscript. ZA-A, BB, and YX contributed to critical revision of the manuscript. ZA-A provided administrative, technical, and material support, and supervision and mentorship. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved. All authors approved the final version of the report. ZA-A is the study guarantor. The corresponding author (ZA-A) attests that all the listed authors meet the authorship criteria and that no others meeting the criteria have been omitted.
Funding: This research was funded by the US Department of Veterans Affairs (ZA-A). The content does not represent the views of the US Department of Veterans Affairs or the US government. The funder had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare support from the US Department of Veterans Affairs for the submitted work; BB reports receiving consultation fees from AstraZeneca; ZA-A reports receiving consultation fees from Gilead Sciences and funding (unrelated to this work) from Tonix Pharmaceuticals; ZA-A reports consulting (uncompensated) for Pfizer; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
ZA-A affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Dissemination to participants and related patient and public communities: The results will be shared via a press release with mainstream media and will also be promoted on twitter (via the account of the corresponding senior author @zalaly). The findings will also be shared with public health agencies and contacts in the US government (the US covid-19 task force and US Centers for Disease Control and Prevention), British Government, and the Canadian Government (the Office of the Chief Science Advisor of Canada). The results will be shared with advocacy groups and will be presented in scientific meetings and at academic institutions.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.