Covid-19 vaccines and treatments: we must have raw data, now
BMJ 2022; 376 doi: https://doi.org/10.1136/bmj.o102 (Published 19 January 2022) Cite this as: BMJ 2022;376:o102
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor,
What surprises me is how can the pharmaceuticals be allowed to withhold clinical trial data? This is not acceptable. It is concerning that data is not being released in a timely manner, which I feel adds further fuel to the anti-vaccination movement. The data must be released in its entirety including any flaws or inconsistencies of any kind. The information is essential for several reasons including: the appropriate scientific scrutiny of the data to help develop future knowledge; to have confidence and to evaluate the full safety profiles of the vaccines, and to engender much needed trust within the pharmaceuticals who are potentially profiteering during a time of global death and turmoil. It is vital in the pursuit of science that we have full access to any available clinical trail data. It is a straightforward task to provide basic trial information, but without seeing the raw data how can a clinical trail be scrutinised? We cannot allow a dangerous precedent. In the long run, these actions will only cast greater suspicion on why the data is being kept from us. In the long run it is not only harmful to scientific discovery, it is harmful for efforts to control and minimise the impact of the COVID-19 pandemic. In particular, it risks a louder anti-vaccination movement. It is sad to say that in the end this could cause lowered support for future vaccination campaigns, which will result in more needless deaths causing greater turmoil to more families.
Competing interests: No competing interests
Dear Editor,
I want to commend Dr Peter Doshi in calling for transparency in clinical trial reporting and in calling for the release of raw data. Since early December I have been communicating with the NEJM on what appears to be a discrepancy in the numbers of people discontinued from the Pfizer COVID mRNA vaccine’s initial trial publication (1) to those published in the 6 month follow up publication (2). There are inconsistencies in the numbers reported on the flow diagrams, in the text of the paper and in the supplemental information in tables S3 of both publications. This confusion makes it hard to reconstruct the conduct of the clinical trial, its data or the conclusions drawn from them.
Initially I attempted to contact the corresponding authors listed on both papers. I was unable to. I later found out the corresponding authors, both Pfizer employees, were no longer with Pfizer. This left both papers with no corresponding author. There should be a policy on reassigning corresponding authorship if one is leaving the institution that published the study. In our digital age this correction or update is very simple.
In bringing the inconsistency of data reporting to the NEJM’s attention they have been able to communicate with someone at Pfizer and have conveyed a limited explanation of the confusing reporting which includes the re-entry of withdrawn individuals into the trial, reclassification of ineligible and withdrawn individuals. There is nothing in the clinical trial protocol that defines how enrollee reentry after withdrawal should be done. Repeated requests for an independent peer review of all the data reported in both papers plus their supplemental information has not, to date, been granted.
The International Committee of Medical Journal Editors and the Committee on Publication Ethics (3) as well as CONSORT 2010 (4) offer guidance on clarity of reporting and how to approach inaccuracies in publications. The use of more detailed flow diagrams to map out and explain the study changes may help. But more importantly perhaps it is time to come to a consensus on all published study data transparency and sharing.
The integrity of medical research, science and public health has suffered enormously during this pandemic. We must rebuild that trust by vowing to be fully transparent and in committing to sharing of clinical trial raw data in addition to adhering to clear data reporting and interpretation within the body of the publication and in the supplemental information attached to it.
(1) Polack F P, Thomas S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine December 31, 2020 383(27):2603
(2) Thomas S, Moreira E, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine through 6 Months. New England Journal of Medicine November 4, 2021 385(19):1761
(3) Publishing and Editorial Issues Related to Publication in Medical Journals. http://www.icmje.org/recommendations/browse/publishing-and-editorial-iss...
(4) Moher D, Hopewell S, et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomized trials. BMJ 2010;340:c869
Competing interests: No competing interests
Dear Editor
May I belatedly cite the ONS report of 20 December 2021, which the BMJ Covid research section pointed up. I so wish I had spotted it earlier, as it would have saved me days of tortured analysis of all-cause mortality vs Vaccination.
Resolution : "I must read my BMJ more attentively !" ..
ONS comprehensively deals with all the "raw data" issues raised by Martin Neil et al., much in line with my own surmises.
"We must have raw data, now", but we really do need peer-reviewed trusted experts to analyse and interpret it for us!
I am thoroughly reassured that Covid-19 vaccination is in practice, as well as in controlled trials, by far a safer decision than non-vaccination, for adults.
References:
[1] ONS 20 Dec 2021. Deaths involving COVID-19 by vaccination status, England: deaths occurring between 1 January and 31 October 2021. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarri...
[2] Neil, Martin. Fenton, Norman. et. al., (2022/01/12); Latest statistics on England mortality data suggest systematic mis-categorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination. Queen Mary London University. United Kingdom. DOI: 10.13140/RG.2.2.28055.09124
Competing interests: No competing interests
Dear Editor
I am grateful to a Traditional Irish Musician (U Magee) for pointing out in a rapid response, what the medical readers ought to see and discuss.
Competing interests: No competing interests
Dear Editor
In answer to Ruth Sharratt (rr 26th Jan), I had stated (my rr 26th Jan):
"ONS source Table 3 [ref 3]. This clearly shows lower ASMRs for vaccinated persons (vs. unvaccinated) for every month from January to October 2021"
I should have been more clear that I was referring to EVER vaccinated (Block 4) vs UNvaccinated (Block 1).
It is true that Block 2 (less than 21 days..) and Block 3 (21 days or more) do have some ASMRs higher than in Block 1 (unvaccinated). Singling out October in Block 2 looks to be selective. UKHSA sets out a fuller analysis and discussion [2, 3]. In particular, they stress the perils of bias, unrepresentativenes, and over-interpretation of raw data [4] having been taken to task [5, 6] about its propensity to mislead, by the UK Office of the Statistics Regulator.
In a thoroughgoing and fascinating interrogation, Martin et al [1] point up several anomalous features in the dataset:-
In each group the non-Covid mortality rates in the three different categories of vaccinated people fluctuate in a wild, but consistent way, far removed from the expected historical mortality rates.
• Whereas the non-Covid mortality rate for unvaccinated should be consistent with historical mortality rates (and if, anything slightly lower than the vaccinated non-Covid mortality rate) it is not only higher than the vaccinated mortality rate, but it is far higher than the historical mortality rate.
• In previous years each of the 60-69, 70-79 and 80+ groups have mortality peaks at the same time during the year (including 2020 when all suffered the April Covid peak at the same time). Yet in 2021 each age group has non-Covid mortality peaks for the unvaccinated at a different time - namely, the time that vaccination rollout programmes for those cohorts reach a peak.
• The peaks in the Covid mortality data for the unvaccinated are inconsistent with the actual Covid wave.
Martin et al. posit some possible explanations, including systematic miscategorisation, delayed or non-reporting, under-estimation, and incorrect population selection.
I would suggest two more straightforward explanations:-
Firstly, the "population change" is straightforwardly explained by the ONS 'mid-year estimate' methodology. UKHSA discusses the pros and cons of using NIMS to calculate the denominator populations.
Secondly, just as UKHSA caveats suggest, I believe it very likely that the mortality anomalies can be largely explained by different behaviours of the people and their vaccinators, rather than any statistical sophistry. I believe it highly likely that a large proportion of people expected or expecting to die were allocated "not for vaccination". This, together with the instruction to vaccinators to exclude young healthy people from the early priority groups, readily explains why peaks of "non-covid mortality" in the UNVACCINATED, matches exactly the vaccination programme schedule. There is then no need to invoke "systematic miscategorisation of deaths between the different groups of unvaccinated and vaccinated."
References:
[1] Neil, Martin. Fenton, Norman. et. al., (2022/01/12); Latest statistics on England mortality data suggest systematic mis-categorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination. Queen Mary London University. United Kingdom. DOI: 10.13140/RG.2.2.28055.09124
[2] https://assets.publishing.service.gov.uk/government/uploads/system/uploa...
[3] https://www.gov.uk/government/publications/phe-monitoring-of-the-effecti...
[4] https://ukhsa.blog.gov.uk/2021/11/02/transparency-and-data-ukhsas-vaccin...
[5] https://osr.statisticsauthority.gov.uk/communicating-data-is-more-than-j...
[6] eugyppius. UKHSA Efficacy Stats Death Watch: Week 44. Nov 5, 2021.
Competing interests: No competing interests
After reading this editorial, I almost stood up and applauded it. Now, going on three years after covid-19, it’s long past due for medical professionals to speak with a collective voice about the stonewalling, delay, and denial pharmaceutical corporations have been engaging in with regard to the release of their covid vaccine trial data. The authors' words spoke directly to the silent frustration millions of physicians like me have been feeling over the same issue.
In the absence of alternative viewpoints arising from analysis of the complete data, the accuracy and credibility of future covid research are damaged. At the same time, physicians and patients will not have access to critical reviews of the studies that allow them to make fully educated decisions with regard to informed consent. Without objective scrutinization of the complete covid vaccine trial data, self-funded published studies and press releases are merely advertising and nothing more.
It's shocking to me that while transparency remains unenforceable or optional for pharmaceutical corporations, hundreds of millions of people have been mandated by their governments and employers to take a vaccine about which the long-term effects we know almost nothing. Forcing anyone to choose between their job and a vaccine that remains shrouded in corporate secrecy is a gross violation of medical ethics, public trust, and personal freedom.
Without new legislative reforms mandating full transparency, we should be reluctant to trust pharmaceutical corporations that have a history of lost lawsuits and fraud convictions. These companies have a well-earned reputation for putting profits before patients, which is why when it comes to trusting them I say beware; the best predictor of future behavior is past behavior. We've been here before.
As such, I feel it's time for the necessary regulatory agencies to mandate data transparency for pharmaceutical corporations because they have proven time and again to not be acting in the service of medicine but their own interests. Until the full data are available for these vaccines, no one can legitimately make any definitive statement regarding their efficacy or safety. So under these difficult circumstances, perhaps the best thing to prescribe with regard to the covid-19 vaccines is caution.
Habib Sadeghi
Physician
Private Practice
Agoura Hills, CA
Competing interests: No competing interests
Dear Editor,
In response to Dr Sam Lewis's comment that ASMR rates are lower for the vaccinated than the unvaccinated, I am confused as to what table he is referring to.
Using the latest ONS data (which only goes to end of October 2021) both Table 2 (non-Covid19 deaths) + All cause deaths (table 3) show the ASMR for October of those who have have had one dose or less than 21 days since 2nd dose, the rates are much higher than in the unvaccinated. Indeed the ASMR of those more than 21 days since 1 dose are more than double the unvaccinated. This surely should be a matter of concern. (It is reflected in other data). I know that Fenton and Neil question the fundamental validity of the data. I do though wonder if this is survival bias. Given what we know from data such as VAERS, the likeliest period for a severe adverse affect is within 7 days of inoculation. It could be that the lower ASMRs for more than 21 days after the second dose are simply due to the fact that if you survive the 1st/ 2nd dose then you're probably going to survive.
I could be misinterpreting the data, but I can't see evidence that vaccination improves survival.
Have a look at the background data
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarri...
We certainly need clarity of data and consistency in definitions, collection and openness in presentation so that eg presenting the data for >14 days after vaccination and so distorting the results does not happen. If vaccines are safe and effective, why does government and the Pharmaceutical industry have to present the data in at best confusing ways.
Competing interests: No competing interests
Dear Editor
I am considerably more reassured in my search for All-cause-mortality comparisons following Covid19 vaccination. I thank Sarah Caul, who pointed me to the ONS source Table 3 [ref 3]. This clearly shows lower ASMRs for vaccinated persons (vs. unvaccinated) for every month from January to October 2021.
I also thank Paul G Champion for furnishing the two papers [refs 1, 2] which interrogate the available data rigorously, and quite properly question its robustness and accuracy. Nonetheless, Martin et al [1] do confirm specifically for all older age brackets, on their page 5, the ONS conclusion that All-cause mortality is lower in the vaccinated. Beattie [2], despite implying a "causal impact" of vaccination upon increased Covid case rates and mortality rates, did not diminish my sense of confidence in Covid19 vaccination. Beattie certainly shows an association between INCREASED case rates, and Vaccination rates, country by country. But given the 'ecological' nature of this cohort comparison, it seems to me equally likely that his explanation could be reversed, such that increasing case rates and deaths lead to increased vaccination. Association is not necessarily Cause.
The BMJ campaign for full disclosure, transparency, and peer review, must continue !
References:
[1] Neil, Martin. Fenton, Norman. et. al., (2022/01/12); Latest statistics on England mortality data suggest systematic mis-categorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination. Queen Mary London University. United Kingdom. DOI: 10.13140/RG.2.2.28055.09124
[2] Beattie, Kyle A. (2021/11/15); Worldwide Bayesian Causal Impact Analysis of Vaccine Administration on Deaths and Cases Associated with COVID-19: A BigData Analysis of 145 Countries. Department of Political Science University of Alberta Alberta, Canada. DOI: 10.13140/RG.2.2.34214.65605
[3] ONS https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarri...
Competing interests: No competing interests
Dear Editor
One hopes that within short order, other journals will follow BMJ’s lead in demanding the same access to data as is considered normal in the other disciplines of science.
Sam Lewis said here (RR 22 January) that he would be more reassured “to know that TOTAL mortality was also lower after vaccination”. Whilst I cannot offer the reassurance which he seeks, I think he et. al., may find these two statistical papers thought provoking. The first has been done by a risk management team whose lead authors were puzzled by the apparent discrepancies between the figures provided by the Office for National Statistics and the Government's figures provided for public consumption. These statisticians therefore, took the all-cause mortality figures as their base line. This avoided getting tied up in knots of collimating exact and varying vaccine statuses, whereas the status of people once deceased does not change. [1]
The second paper I found on the pre-print server on the day it was uploaded. Not being peer reviewed, I down loaded it to digest at leisure and to look for obvious non sequiturs in the methodology. I was impressed by the thoroughness with which the author had scraped what little world data is available in an effort to tease out what effect that covid and vaccinations may have had on overall mortality in 145 countries. This paper appears to show a common pattern between covid case rates, vaccination rates, and overall mortality. In doing so it echos the analysis done in the UK too. [2]
Whilst both papers detail the limitations of their analysis and thus causality behind all the numbers is still left open to some debate. Who, after reading these papers in their entirety, could possibly deny, we should have had all the applicable data by yesterday? Not tomorrow or some time never — we need it now.
References:
[1] Neil, Martin. Fenton, Norman. et. al., (2022/01/12); Latest statistics on England mortality data suggest systematic mis-categorisation of vaccine status and uncertain effectiveness of Covid-19 vaccination. Queen Mary London University. United Kingdom. DOI: 10.13140/RG.2.2.28055.09124
[2] Beattie, Kyle A. (2021/11/15); Worldwide Bayesian Causal Impact Analysis of Vaccine Administration on Deaths and Cases Associated with COVID-19: A BigData Analysis of 145 Countries. Department of Political Science University of Alberta Alberta, Canada. DOI: 10.13140/RG.2.2.34214.65605
Competing interests: No competing interests
Cuban vaccines have the capacity to generate antibodies against the omicron variant of COVID-19.
Dear Editor
Preliminary studies developed from the Pedro Kourí Institute of Tropical Medicine (IPK) in vaccinated with Soberana and Abdala show high percentages of seroconversion against omicron. Or what is the same: our vaccines have the capacity to generate antibodies against this new variant of the virus.
The effectiveness against omicron in the biological Soberana 02 and Abdala is seen at 90 percent, while in Soberana 01 is seen at 100 percent.
These results confirm that vaccination schedules, as well as booster doses, are a successful strategy against the COVID-19 pandemic.
Meanwhile, in the coming days, Cuba will formally present the dossier of its Soberana 02 and Soberana 01 and 02 vaccines to the World Health Organization (WHO) to seek the endorsement of that body.
In recent weeks, the Mexican health authorities gave the go-ahead for Abdala. The island's vaccines were also exported to Venezuela, Nicaragua, Iran and Vietnam.
Cuba completed the vaccination scheme for more than 90 percent of its population with its three products, while it began a reinforcement program with a view to stopping the advance of the omicron variant, which arrived on the island after it reopened its economy and its international flights last November.
Kind Regards
Competing interests: No competing interests