Covid-19: 146 researchers raise concerns over chloroquine study that halted WHO trialBMJ 2020; 369 doi: https://doi.org/10.1136/bmj.m2197 (Published 02 June 2020) Cite this as: BMJ 2020;369:m2197
A group of international researchers has raised methodological and data integrity concerns about a recent study on hydroxychloroquine and chloroquine in covid-19 patients, which led to the World Health Organization halting part of its trial.
The observational study, published in the Lancet,1 reported that inpatients with covid-19 who were treated with chloroquine or hydroxychloroquine (with or without a macrolide) showed an increased risk of dying in hospital and of de novo ventricular arrhythmia than patients who did not receive the drugs.2
WHO has been testing potential covid-19 treatments through its Solidarity Trial, involving over 400 hospitals in 35 countries. After the publication of the Lancet paper, which included patients from six continents, WHO announced that the hydroxychloroquine arm of the trial had been halted while the Data Safety Monitoring Board reviewed the data.
In an open letter to the study’s authors and the journal’s editor, Richard Horton, however,3 146 clinicians, medical researchers, statisticians, and ethicists have now highlighted 10 main concerns about the research.
Further error checking
The concerns outlined in the open letter included “inadequate adjustment for known and measured confounders,” the lack of an ethics review, and no data/code sharing—despite the Lancet being a signatory to the Wellcome statement on data sharing for covid-19 studies.4
The letter also highlighted that there was “no mention of the countries or hospitals that contributed to the data source and no acknowledgments to their contributions” and that a request to the study authors for this information had been denied.
Regarding findings from Australia, the letter said that the data were “not compatible with government reports,” as they showed “too many cases for just five hospitals” and more hospital deaths than had occurred in the entire country during the study period.
“Surgisphere (the data company) has since stated this was an error of classification of one hospital from Asia. This indicates the need for further error checking throughout the database,” the letter said.
It added that the data from Africa indicated that nearly 25% of all covid-19 cases and 40% of all deaths had occurred in Surgisphere associated hospitals, which had sophisticated electronic patient data recording and patient monitoring. “Both the numbers of cases and deaths, and the detailed data collection, seem unlikely,” the letter stated.
Surgisphere Corporation—which focuses on healthcare data analytics and medical education and has developed a diagnostic tool for covid-19—was founded by Sapan Desai, one of the authors of the study. It provided some funding and statistical help for the research.
To tackle the issues, the letter’s signatories have called for Surgisphere to provide details on data provenance, for WHO to independently validate the analysis and conclusions, and for all data sharing agreements to be made available so that they can check that “any mined data was legally and ethically collected and patient privacy aspects respected.”
They have also asked the Lancet to make openly available the peer review comments that led to this manuscript being accepted for publication.
The Lancet has since posted a correction to the paper (29 May), but it said that the findings had not changed.5 The correction said, “In the first paragraph of the Results section, the numbers of participants from Asia and Australia should have been 8101 (8.4%) and 63 (0.1%), respectively. One hospital self-designated as belonging to the Australasia continental designation should have been assigned to the Asian continental designation.”
The study had previously stated that there were 7555 participants (7.9%) from Asia and 609 (0.6%) from Australia.
The correction continued, “The appendix has also been corrected. An incorrect appendix table S3 was included, originally derived from a propensity score matched and weighted table developed during a preliminary analysis. The unadjusted raw summary data are now included. There have been no changes to the findings of the paper.”
Both the lead author—Mandeep Mehra, medical director of the Brigham and Women’s Hospital Heart and Vascular Center—and Surgisphere also pointed to these changes.6
However, James Watson, statistician at the University of Oxford Centre for Tropical Medicine and Global Health and signatory to the letter, said that the correction had dealt with only one of the 10 points of concern.
He told The BMJ, “I think that many people are disappointed with the Lancet’s handling of the post-publication review of the hydroxychloroquine paper. The Lancet appears to be stating that authors have responded appropriately—which I think they have not—instead of making a statement that they will investigate our concerns.”
There is also concern over the effect that this situation will have on ongoing hydroxychloroquine trials and therefore the ability to get answers on the potential covid-19 treatment.
Buddha Basnyat, fellow letter author and director of the Oxford University Clinical Research Unit-Nepal, said, “It is clearly going to have a negative impact on randomised control trials being conducted or in the process of being conducted worldwide, especially in low and middle income countries.”
This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage