Re: Risk prediction of covid-19 related death and hospital admission in adults after covid-19 vaccination: national prospective cohort study
Dear Editor
In a recent issue of the BMJ, Hippisley-Cox et al. [1] published a study reporting two new risk prediction algorithms to estimate the risk of COVID-19 related mortality and hospital admission in UK adults after vaccination. The QCovid tool had previously been helpful in predicting risk of COVID-19 related death or hospital admission based on individual characteristics to identify patients at high risk of severe outcomes [2]. The use of model-based statistical tools such as QCovid is, and should continue to be, an important resource to aid in the early identification of populations at risk. However, we have several concerns about how the findings related to the population of individuals with Down syndrome were presented and contextualized in the paper.
From a statistical perspective, we are concerned that it is not possible to derive reliable and robust hazard ratios (HR) and 95% confidence intervals (95% CIs) for groups with extremely small sample sizes and almost no outcome events (hospitalizations and deaths), as it was the case for individuals with Down syndrome. Table 1 of the paper by Hippisley-Cox et al. [1] shows that the study included 3,963 vaccinated individuals with Down syndrome (0.06% of the total study population) and, most importantly, only between 0 and 4 of these individuals were hospitalized or died (assuming the “-“ is defined as <5 in Table 1, as it was in the supplementary tables).
HR is defined as the ratio of the hazard rates between two hazards model-fitted survival curves. In calculating HR, the researcher’s decision about when to follow up is arbitrary and may lead to significant variations in reported hazard ratios. Hippisley-Cox et al. [1] defined it as “at the start of follow-up at 14 days after the first dose”. Using the first vaccine dose as the follow-up point assumes that all study populations would respond similarly to the first and second doses of the vaccine, which is not necessarily true. Additionally, both the Oxford-AstraZeneca and Pfizer-BioNTech vaccines require two vaccine doses to be fully effective. Therefore, it is not surprising that a small number of individuals with Down syndrome were hospitalized or died with COVID-19 between the first and the second dose of the vaccine. More importantly are the infections that occurred 14 days or more after the second dose. However, these data were not clearly presented in the article and the conclusions were mainly based on infections that occurred after the first dose of the vaccine.
The pattern of HR for death and for hospitalization for people with Down syndrome is also difficult to interpret. For example, the authors showed that the calculated HR (95% CI) for COVID deaths after vaccination (fig. 1 in the paper) was 8.07 (3.34 to 19.54) for those with kidney transplantation and 12.68 (4.68 to 34.38) for those with Down syndrome. In contrast, the HR (95% CI) for COVID hospitalizations (fig. 2) were 12.82 (7.65 to 21.47) and 2.55 (0.63 to 10.28), respectively, for these same groups. These numbers indicate that the HR of individuals with Down syndrome being hospitalized is not significantly larger than that of those in the general vaccinated population, which is in clear contrast to the results for mortality, while in those with kidney transplants the two HR values were comparable, as would be reasonably expected.
Some of the issues presented here derive from the small numbers with outcomes of interest. Although specific database sizes should be mentioned (in this case 6,952,440), much greater emphasis should always be given to the actual number of outcome events (which, in this case were hospitalizations and deaths). The data indicate that just 81 deaths (4.0%) and 71 admissions (3.7%) in total occurred 14 days or more after the second vaccine dose. The authors were careful to list the limitations of their work. Unfortunately, these seldom are used in the reporting of results in the lay media, which can cause harm. Often the media focus on aspects of studies that are not the key findings of the study, for example, the large number of participants in the databases from which the data are drawn is emphasized, whereas the actual number of events are not mentioned; this does not invalidate the use of the model, but if emphasized beyond the total database size, should help the reader (including science writers) to put things in perspective.
Given these limitations, it is unfortunate that Hippisley-Cox et al. [1] included a hypothetical example of an elderly person with Down syndrome who has been partially rather than fully vaccinated, and with an unusually high BMI for a person with Down syndrome of this age. Presumably these extreme parameters were chosen to accentuate risk estimates, but this example is not representative of the average person with Down syndrome. The inclusion of a case of a young and fully-vaccinated individual with Down syndrome would have been more informative and helpful for families and carers.
We would argue that, in the current environment, it behooves the scientist to anticipate how journalists may interpret and communicate findings. This paper has resulted in media coverage that is inaccurately alarming – for example, The Guardian published an article that states that “people with Down’s syndrome had a roughly 13-fold increased risk of death from Covid-19 compared with the general population, even after vaccination, […]” [3], which is just one example of a well-meaning but distorted media coverage. This has raised concern within the Down syndrome community that people remain at exceptionally high risk even when fully vaccinated, and may question the value of vaccination in this population.
Prioritization of vaccination for people with Down syndrome has been hard-won [4], and our large, collective experience suggest that the vaccines are well tolerated in individuals with Down syndrome. Including Down syndrome as a high-risk population in large studies such as this one helps to make sure the population is not ignored or left out of vaccination and pandemic response efforts. Nonetheless, properly contextualizing findings when published in a leading medical journal is also clearly warranted.
Andre Strydom, Professor in Intellectual Disabilities, King's College London, and president, Trisomy 21 Research Society, IoPPN, 16 de Crespigny Park. SE5, London, UK; Professor Alberto Costa, Case Western Reserve University and Chair T21RS Clinical committee; Dr Anke Hüls, Emory University (Environmental health and epidemiology), USA; Professor Stephanie Sherman, Emory University and Co-Chair T21RS Clinical committee, USA; Professor Yona Lunsky, University of Toronto, Canada; on behalf of the The Trisomy 21 Research Society (T21RS) COVID-19 Task Force with the support of the Down syndrome medical interest group (USA), Down syndrome medical interest group (UK), LuMind IDSC foundation, National Task Group on Intellectual Disabilities and Dementia Practices (NTG), Down Syndrome Education International (DSEI), and Down España.
1. Hippisley-Cox J, Coupland CA, Mehta N, Keogh RH, Diaz-Ordaz K, Khunti K, Lyons RA, Kee F, Sheikh A, Rahman S, Valabhji J, Harrison EM, Sellen P, Haq N, Semple MG, Johnson PWM, Hayward A, Nguyen-Van-Tam JS. Risk prediction of covid-19 related death and hospital admission in adults after covid-19 vaccination: national prospective cohort study. BMJ. 2021;374:n2244. doi: 10.1136/bmj.n2244.
2. Hippisley-Cox J, Clift AK, Coupland CAC, et al. Protocol for the development and evaluation of a tool for predicting risk of short-term adverse outcomes due to COVID-19 in the general UK population. medRxiv 2020:2020.06.28.20141986-2020.06.28.86. doi:10.1101/2020.06.28.20141986
3. https://www.theguardian.com/world/2021/sep/18/people-with-chronic-condit...
4. https://www.t21rs.org/covid-19/
Competing interests:
No competing interests
10 October 2021
Andre Strydom
Professor in Intellectual Disabilities
Professor Alberto Costa, Case Western Reserve University and Chair T21RS Clinical committee; Dr Anke Hüls, Emory University (Environmental health and epidemiology), USA; Professor Stephanie Sherman, Emory University and Co-Chair T21RS Clinical committee, USA; Professor Yona Lunsky, University of Toronto, Canada; on behalf of the The Trisomy 21 Research Society (T21RS) COVID-19 Task Force with the support of the Down syndrome medical interest group (USA), Down syndrome medical interest group (UK), LuMind IDSC foundation, National Task Group on Intellectual Disabilities and Dementia Practices (NTG), Down Syndrome Education International (DSEI), and Down España
King's College London, and president, Trisomy 21 Research Society
Rapid Response:
Re: Risk prediction of covid-19 related death and hospital admission in adults after covid-19 vaccination: national prospective cohort study
Dear Editor
In a recent issue of the BMJ, Hippisley-Cox et al. [1] published a study reporting two new risk prediction algorithms to estimate the risk of COVID-19 related mortality and hospital admission in UK adults after vaccination. The QCovid tool had previously been helpful in predicting risk of COVID-19 related death or hospital admission based on individual characteristics to identify patients at high risk of severe outcomes [2]. The use of model-based statistical tools such as QCovid is, and should continue to be, an important resource to aid in the early identification of populations at risk. However, we have several concerns about how the findings related to the population of individuals with Down syndrome were presented and contextualized in the paper.
From a statistical perspective, we are concerned that it is not possible to derive reliable and robust hazard ratios (HR) and 95% confidence intervals (95% CIs) for groups with extremely small sample sizes and almost no outcome events (hospitalizations and deaths), as it was the case for individuals with Down syndrome. Table 1 of the paper by Hippisley-Cox et al. [1] shows that the study included 3,963 vaccinated individuals with Down syndrome (0.06% of the total study population) and, most importantly, only between 0 and 4 of these individuals were hospitalized or died (assuming the “-“ is defined as <5 in Table 1, as it was in the supplementary tables).
HR is defined as the ratio of the hazard rates between two hazards model-fitted survival curves. In calculating HR, the researcher’s decision about when to follow up is arbitrary and may lead to significant variations in reported hazard ratios. Hippisley-Cox et al. [1] defined it as “at the start of follow-up at 14 days after the first dose”. Using the first vaccine dose as the follow-up point assumes that all study populations would respond similarly to the first and second doses of the vaccine, which is not necessarily true. Additionally, both the Oxford-AstraZeneca and Pfizer-BioNTech vaccines require two vaccine doses to be fully effective. Therefore, it is not surprising that a small number of individuals with Down syndrome were hospitalized or died with COVID-19 between the first and the second dose of the vaccine. More importantly are the infections that occurred 14 days or more after the second dose. However, these data were not clearly presented in the article and the conclusions were mainly based on infections that occurred after the first dose of the vaccine.
The pattern of HR for death and for hospitalization for people with Down syndrome is also difficult to interpret. For example, the authors showed that the calculated HR (95% CI) for COVID deaths after vaccination (fig. 1 in the paper) was 8.07 (3.34 to 19.54) for those with kidney transplantation and 12.68 (4.68 to 34.38) for those with Down syndrome. In contrast, the HR (95% CI) for COVID hospitalizations (fig. 2) were 12.82 (7.65 to 21.47) and 2.55 (0.63 to 10.28), respectively, for these same groups. These numbers indicate that the HR of individuals with Down syndrome being hospitalized is not significantly larger than that of those in the general vaccinated population, which is in clear contrast to the results for mortality, while in those with kidney transplants the two HR values were comparable, as would be reasonably expected.
Some of the issues presented here derive from the small numbers with outcomes of interest. Although specific database sizes should be mentioned (in this case 6,952,440), much greater emphasis should always be given to the actual number of outcome events (which, in this case were hospitalizations and deaths). The data indicate that just 81 deaths (4.0%) and 71 admissions (3.7%) in total occurred 14 days or more after the second vaccine dose. The authors were careful to list the limitations of their work. Unfortunately, these seldom are used in the reporting of results in the lay media, which can cause harm. Often the media focus on aspects of studies that are not the key findings of the study, for example, the large number of participants in the databases from which the data are drawn is emphasized, whereas the actual number of events are not mentioned; this does not invalidate the use of the model, but if emphasized beyond the total database size, should help the reader (including science writers) to put things in perspective.
Given these limitations, it is unfortunate that Hippisley-Cox et al. [1] included a hypothetical example of an elderly person with Down syndrome who has been partially rather than fully vaccinated, and with an unusually high BMI for a person with Down syndrome of this age. Presumably these extreme parameters were chosen to accentuate risk estimates, but this example is not representative of the average person with Down syndrome. The inclusion of a case of a young and fully-vaccinated individual with Down syndrome would have been more informative and helpful for families and carers.
We would argue that, in the current environment, it behooves the scientist to anticipate how journalists may interpret and communicate findings. This paper has resulted in media coverage that is inaccurately alarming – for example, The Guardian published an article that states that “people with Down’s syndrome had a roughly 13-fold increased risk of death from Covid-19 compared with the general population, even after vaccination, […]” [3], which is just one example of a well-meaning but distorted media coverage. This has raised concern within the Down syndrome community that people remain at exceptionally high risk even when fully vaccinated, and may question the value of vaccination in this population.
Prioritization of vaccination for people with Down syndrome has been hard-won [4], and our large, collective experience suggest that the vaccines are well tolerated in individuals with Down syndrome. Including Down syndrome as a high-risk population in large studies such as this one helps to make sure the population is not ignored or left out of vaccination and pandemic response efforts. Nonetheless, properly contextualizing findings when published in a leading medical journal is also clearly warranted.
Andre Strydom, Professor in Intellectual Disabilities, King's College London, and president, Trisomy 21 Research Society, IoPPN, 16 de Crespigny Park. SE5, London, UK; Professor Alberto Costa, Case Western Reserve University and Chair T21RS Clinical committee; Dr Anke Hüls, Emory University (Environmental health and epidemiology), USA; Professor Stephanie Sherman, Emory University and Co-Chair T21RS Clinical committee, USA; Professor Yona Lunsky, University of Toronto, Canada; on behalf of the The Trisomy 21 Research Society (T21RS) COVID-19 Task Force with the support of the Down syndrome medical interest group (USA), Down syndrome medical interest group (UK), LuMind IDSC foundation, National Task Group on Intellectual Disabilities and Dementia Practices (NTG), Down Syndrome Education International (DSEI), and Down España.
1. Hippisley-Cox J, Coupland CA, Mehta N, Keogh RH, Diaz-Ordaz K, Khunti K, Lyons RA, Kee F, Sheikh A, Rahman S, Valabhji J, Harrison EM, Sellen P, Haq N, Semple MG, Johnson PWM, Hayward A, Nguyen-Van-Tam JS. Risk prediction of covid-19 related death and hospital admission in adults after covid-19 vaccination: national prospective cohort study. BMJ. 2021;374:n2244. doi: 10.1136/bmj.n2244.
2. Hippisley-Cox J, Clift AK, Coupland CAC, et al. Protocol for the development and evaluation of a tool for predicting risk of short-term adverse outcomes due to COVID-19 in the general UK population. medRxiv 2020:2020.06.28.20141986-2020.06.28.86. doi:10.1101/2020.06.28.20141986
3. https://www.theguardian.com/world/2021/sep/18/people-with-chronic-condit...
4. https://www.t21rs.org/covid-19/
Competing interests: No competing interests