“Excess deaths” is the best metric for tracking the pandemic
BMJ 2022; 376 doi: https://doi.org/10.1136/bmj.o285 (Published 04 February 2022) Cite this as: BMJ 2022;376:o285Read our latest coverage of the coronavirus pandemic
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor,
I am in complete agreement with the title and content of your editorial, stating that “Excess deaths” is the best metric for tracking the pandemic and that is ‘more informative and accurate than covid-19 deaths or cases’. Using the exact same reasoning, I claim here that ‘excess lives saved’ is the best metric for tracking vaccine efficacy. In a paraphrase of your words, I suggest that comparing all-cause mortality in vaccinated vs unvaccinated individuals is more informative and accurate than merely reporting on deaths due to covid-19, yet this metric is hardly ever reported in trials describing vaccine efficacy or safety. Here I will review literature relating to the Pfizer-Biontech BNT162b2 mRNA Covid-19 Vaccine, although such claims can be generalized to all Covid-19 vaccines.
The primary end points of the three pivotal prospective randomized trials assessing this vaccine in the different age groups (namely in people over 16 years, in 12-15 year-olds and in 5-11 year-olds) were efficacy of the vaccine against laboratory-confirmed Covid-19 and safety [1,2,3]. To date, long-term survival data was published only for the initial adult cohort; this data failed to demonstrate any differences in overall survival between the vaccinated and non-vaccinated groups (15 vs 14 deaths in both groups, respectively, at time of data analysis) [4].
Several retrospective analyses have been published throughout 2021 and 2022 ([5,6] to quote just two), many of which were published in journals of the highest impact, reporting on covid-related health outcomes such as covid-related deaths and hospitalizations. The majority, if not all, of these manuscripts did not report all-cause mortality of vaccinated vs unvaccinated subjects. Moreover, it has become customary, in these retrospective reports, not to provide any outcome measurements (including survival) of patients within 7-14 days of vaccine administration. Whereas the alleged justification for this omission is that the vaccine is expected to start being effective only a few days following its administration, clearly this period of time cannot be ignored in terms of mortality. Indeed, the guidelines of ‘Good Clinical Practice’ (GCP) mandate that every adverse event that occurs to a subject at any time point on trial must be meticulously documented and eventually reported [7]. Whereas GCP mainly relates to prospective clinical trials, similar scrutiny should have been used in these pivotal retrospective trials as well. A ‘limbo’ period in which outcomes of subjects are not reported is methodologically wrong and unethical, and should not have been acceptable by the scientific comunity.
Most striking is the fact that in a retrospective matched case-control study studying vaccine safety by assessing more than 20 potential side-effects of vaccination, the most clinically-significant outcome – namely, all-cause mortality – has not been reported [8]. This is peculiar, as the authors of this paper have clearly collected the data on death, as it was defined as one of the potential outcomes leading to thetermination of matching between the vaccinated and unvaccinated ‘pairs’.
The editorial rightfully claims that the metric of ‘covid-related death’ is not accurate. The exact claim is similarly true when it comes to assessing the metric ‘prevention of covid-19-related- death’, which is not accurate as well. For instance, if vaccinated individuals were not routinely screened for Covid-19 upon their admission to the hospital with non-covid-related symptoms (as has been the case at least in Israel up until fall 2021), then their subsequent death would not have been attributed to Covid-19, in contrast to an unvaccinated patient presenting with identical non-covid-related symptoms, who would have been screened upon admission and potentially found to be Covid-19-positive.
In order to be able to truly assess vaccine efficacy and its benefit-risk ratio, we urge the academia to insist on the reporting of all-cause mortality in vaccinated vs non-vaccinated subjects, with the definition of a vaccinated individual being a person immediately following his/her first vaccination, without any ‘limbo’ period. Similarly, we request that this data be provided for both the short term and for the long term. This will allow to probe out potential long-term effects of the vaccine (be them beneficial or detrimental) on the general health status of vaccinated individuals, among them overall survival.
References:
1. F.P. Pollac et al. N Engl J Med 2020; 383:2603-2615
DOI: 10.1056/NEJMoa2034577.
2. R. W. Frenck et al. N Engl J Med 2021; 385:239-250
DOI: 10.1056/NEJMoa2107456
3. E. B. Walter et al. N Engl J Med 2022; 386:35-46
DOI: 10.1056/NEJMoa2116298
4. S. J. Thomas et al. N Engl J Med 2021; 385:1761-1773
DOI: 10.1056/NEJMoa2110345
5. Y. M. Bar-On. N Engl J Med 2021; 385:1393-1400
DOI: 10.1056/NEJMoa2114255
6. Y. M. Bar-On. N Engl J Med 2021; 385:2421-2430
DOI: 10.1056/NEJMoa2115926
7. https://database.ich.org/sites/default/files/E6_R2_Addendum.pdf
8. N.Barda et al. N Engl J Med 2021; 385:1078-1090
DOI: 10.1056/NEJMoa2110475
Competing interests: Dr. Leibowitz has previously served as a compensated consutant to Pfizer
Dear Editor,
We believe that excess mortality data can be used as an indicator to describe the actual burden of covid-19, but attention should be paid to the other direction of uncertainty, overestimation. Overestimation of covid-19 mortality may also bring worries to the health system, and negatively affect resource allocation and response to epidemics.
The excess mortality caused by covid-19 are mainly diseases related deaths (lung tissue damage, bronchial complications, and circulatory system problems such as microthrombosis and thrombosis), as well as excessive demand for medical resources, causing mortality among patients with other diseases due to unable or unwilling to get timely treatment in hospital. In addition, mental and emotional disorders and suicides during the lock-down also contributed to the excess mortality. However, the small number of excess deaths in Germany shows that their healthcare system was not overwhelmed by the number of covid-19 cases and that clinical services were provided as part of routine practice and primary care and treatment. No increase in overall mortality was observed in a large city or its nursing homes. [1] Although elective surgeries were postponed due to a shortage of healthcare resources and new rules to stop covid-19's spread, these interventions did not significantly increase deaths among patients waiting for their surgeries. [2] The active numbers and removal due to death from waiting lists for organ transplantation, which relies heavily on intensive care unit capacity, showed the same trend. [3]
More or less part of excess deaths cannot be explained in many countries. For example, there is a mismatch of 74% excess mortality not explained by covid-19 in Peru, and about 25% and 35% in the United States and Spain, respectively. Therefore, the initial hypothetical number of deaths in Robert Koch Institute has not been widely accepted. Although forensic diagnosis can be used to determine whether a disease directly leads to death, for example, a study from the University Medical Center Hamburg-Eppendorf showed that covid-19 were identified as the cause of deaths in 84% infected cases. [4] But before that, it is necessary to have accurate number of excess deaths and those infected with covid-19 in order to estimate the true burden of covid-19 accordingly.
As the vaccination rate increases and more countries chose to live with Covid, the true burden of cannot be underestimated, but it should not be overestimated as well. Besides excess mortality data, the estimation of true number of people who died of covid-19 also requires a complete forensic diagnosis and a reasonable sampling frame to accurately estimate the proportion.
Dong Qu[1], Zhong Huang[2,3], Zhi Qu[4]
Professional affiliations of the authors:
1 Institute of Legal Medicine, Hannover Medical School, 30625, Hannover, Germany
2 Institute of Neuroanatomy and Cell Biology, Hannover Medical School, 30623 Hannover, Germany
3 Center for Systems Neuroscience (ZSN) Hannover, 30559 Hannover, Germany
4 Institute of Epidemiology, Social Medicine, and Health System Research, Hannover Medical School, 30625, Hannover, Germany.
REFERENCES
1 Heudorf U, Müller M, Schmehl C, et al. COVID-19 in long-term care facilities in Frankfurt am Main, Germany: incidence, case reports, and lessons learned. GMS hygiene and infection control 2020;15.
2 Metelmann IB, Busemann A. Elective surgery in times of COVID-19: A two-centre analysis of postponed operations and disease-related morbidity and mortality. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 2020;158:62–65.
3 Eurotransplant. Yearly statistics(25.05.2021) 2021. Available at: https://www.eurotransplant.org/statistics/yearly-statistics/.
4 Universitätsklinikum Hamburg-Eppendorf. Auswertung bestätigt: Therapieumstellung bei COVID-19-Erkrankten reduziert Risiko 2021. Available at: https://www.uke.de/allgemein/presse/pressemitteilungen/detailseite_10432... Accessed October 20, 2021.
Competing interests: No competing interests
Re: “Excess deaths” is the best metric for tracking the pandemic
Dear Editor
All causes mortality (ACM) is evidently the best metric to describe this aspect of public health. The interpretation, however, includes factors that the author did not emphasize.
ACM reflects not only the direct and indirect effects of the viral epidemic, but also the effects of measures undertaken by the health authorities/governments to combat the epidemic and the effects accrued as a result of the public response to these measures. Hence the effects of vaccinations, isolation, wearing masks, avoiding or postponing elective medical checkups and procedures, etc. appear in the same metric and obscure the net direct/indirect effect of the epidemic. The examination of the relative contribution of these two categories of effects is both difficult and often hampered for political reasons.
Competing interests: No competing interests