Re: Search for evidence goes on
The rapid response of Dr Hayden, Dr Treanor and Dr Kaiser (1) highlights concerns about inaccuracies in the recent BMJ feature (2). We would like to take the opportunity to respond to the comments made.
We appreciate that at the time of writing the feature, Dr Treanor firmly denies the involvement of an unnamed medical writer on Treanor (2000), which appeared in JAMA (3).
Dr Treanor told the BMJ on 1st December 2009: “The manuscript in question was written by myself. I can state unequivocally that no ghost writer was involved, and I do not recall ever having contact with anyone from Adis regarding this manuscript or its content. I was definitely not given "key messages" to insert into the manuscript.” He also noted that Drs. Mills and Ward—both Roche employees as stated in the JAMA paper—had input into the manuscript.
The BMJ has also put the allegation of ghostwriting to Roche. Roche has been given three opportunities to deny that the paper at issue, Treanor (2000), used undisclosed writers. They have not done this:
1. We put it to Roche in December 2009 that: “The BMJ also intends to report: a. that an individual, working for a medical communications company called Adis, was involved in the preparation of the above papers without that person’s role being disclosed (i.e. a ‘ghost-writer’”). The papers in question were Treanor (2000) and Nicholson (2000).
Roche responded: “Roche confirms that medical writers were used to help draft some of the above papers. This is neither unusual nor secretive, and is common practice in the scientific community. At the time of writing and submission (2002) (sic), it was not standard practice for professional medical writers to be named on manuscripts.”
2. We also put it to Roche in December 2009 that: “according to one of the medical writers from Adis who worked on the Tamiflu account, Roche’s marketing department involved itself in the preparation of journal articles including Nicholson (2000) and Treanor (2000). Roche’s marketing department, according to the BMJ’s source, obliged writers of scientific papers to include in the papers a series of “key messages” that included emphasising the importance of influenza and the role of Tamiflu in treating”.
Roche responded: “Although common at the time, the term “key messages” is no longer used in the context of publications as it has been frequently misinterpreted. We are not clear if there is a specific inclusion in the manuscript that is considered inappropriate, imbalanced or inaccurate; if so, this has not been made clear to us. Roche also refutes that it obliged writers to include any wording or component of the content: content of any paper involving a medical writer would be developed in conjunction with the authors, and would be reviewed and approved by them before submission, and under their control at all times. We have records verifying that the authors had access to data and provided review and comments on the manuscripts.”
3. In the latest investigation into the evidence base for oseltamivir, the BMJ put a series of questions to Roche. Roche opted to make additional points about the feature in the BMJ. They made a specific point about ghostwriting – which did not include a denial about Treanor (2000) having used unnamed medical writers.
Roche said: “Roche maintains the highest ethical standard in the writing of clinical papers for publication, the interaction with external clinical investigators and authors, and has always clearly disclosed its own financial and expert involvement in studies and publications. During the period of time in question (1999 – 2002) it was common practice for scientific medical writers to provide writing support for publications with the authors having full access to data and full and final review of the publications. Since the introduction in 2003 of the Good Publication Practice guidelines for Pharmaceutical Companies (GPP), Roche has complied with the practice to acknowledge the involvement of professional medical writers.”
Back in 2009, we also asked Dr Treanor about his access to primary data.
On 2nd December 2009, Dr Treanor replied: “The JAMA paper was written by me, based on analysis of primary data conducted by Roche, including supplemental analyses that I requested and that they performed. I did not request the data set because I did not believe it to be necessary and in any case I am not a statistician, don't know how to use SAS, and would not have been able to do anything with it if I did have it. I don't remember that JAMA required me to be able to produce the primary data on request, but in any case they obviously never asked for it. I suppose that if they had, Roche would have supplied it to them, but I am not sure how relevant that is to Dr. Jefferson's [one of the authors of a Cochrane review looking at the effectiveness of Tamiflu] request.”
“I can't imagine that anything that an author signs on submission to a journal about the primary data is meant to last in perpetuity but I didn't save any of that kind of journal correspondence so who knows what I might have signed. If JAMA were to produce the agreement and ask for the primary data (which they haven't) I would guess that Roche would provide it, but I suspect that there would be arguments about whether the agreement still applies so many years after submission, and would probably limit the data they provided to the minimum required to support the paper. I don't know if that would be worth pursuing. As you know, Tom [Jefferson] already asked me about this and then requested the data from Roche, which I guess they declined to provide. But of course, Tom isn't a representative of JAMA.”
Given Roche’s refusal to give the Cochrane Collaboration access to raw data, the BMJ now urges Dr Howard Bauchner, editor in chief of JAMA, to write to the company and ask them to provide the data.
While we are prepared to accept Dr Treanor’s assurances that he was unaware that his paper was ghostwritten, this of course does not mean that it was not. Roche’s evasive answers when asked about this matter only serve to reinforce our concerns.
On the second point, we believe that it was fair and accurate to describe the Kaiser (2003) review that appeared in Archives of Internal Medicine (4) as a Roche funded review, given the extent of the input that Roche employees had into the final analysis and paper. We did not allege that either Professor Hayden or Professor Kaiser received financial compensation for their work on the review.
Co-authors on the paper are Cynthia Wat, Tracy Mills, Paul Mahoney, and Penelope Ward—all Roche employees. As disclosed at the end of the paper: “This study was supported by F. Hoffmann-La Roche Ltd, Basel, Switzerland.”
It is acknowledged on the paper that help with the statistical analysis was “provided by Vivien Mitchell, MSc (Roche Global Development)”.
Furthermore, on 17th November 2009, Professor Kaiser said to the BMJ: “I designed the project, participated [in] the analysis. [The] statistical analysis and data base management were performed by the Roche team, and then I drafted the paper with my colleague Fred Hayden.”
Of note, when they wrote to the BMJ earlier this month, Roche did not correct the description of a “Roche funded review” – even though they took the opportunity to discuss other points made in the article.
There are several other issues at stake with Kaiser (2003), published in the Archives of Internal Medicine, which we will take this opportunity to highlight.
When the Cochrane Collaboration asked both Professor Hayden and Professor Kaiser to make available the data from the trials that had been selected for inclusion in the review (eight of the ten included trials were unpublished), they were unable to do so.
Professor Hayden said on the 14th August 2009: “I have searched but cannot find the original files related to this 2003 publication. Before and again after my 2+ years at WHO in Geneva, I was obliged to move offices at the University several times and downsize. The files appear to have been discarded. My co-author Laurent Kaiser, now professor at the University of Geneva, is copied on this reply, as he may have his own sources. The questions posed by the inquirer are not clear to me, but if original data or unpublished study reports are required, they will likely need to come from Roche, the sponsor of these studies”.
On the 17th August 2009, Dr Kaiser wrote: “I suggest to contact (sic) Roche directly to get access to the files”
He subsequently told the BMJ on 17th November 2009 that “the data management was centralised and I do not have the database here in Geneva”.
Following on from a series of email exchanges, Dr Hayden told the BMJ he had access to raw data. He subsequently forwarded an email he had sent to James Smith, International Medical Leader of Tamiflu at Roche on the 1st December 2009 saying:
“Since neither I nor Laurent have remaining hardcopies of the original datasets, I would ask for written confirmation from you that Roche will provide these to us if requested by us, the editors of Archives of Internal Medicine, or their assignees. Also as requested in my email in early October, I would ask that Roche provide the requested clinical trials data to Dr. Tom Jefferson for his analyses. It is my personal opinion that Roche should make these data available to all legitimate investigators.”
To which Dr Smith responded: “We can confirm that Roche have the original datasets. Roche has made the reports available and is always willing to provide data in response to legitimate requests.”
To ensure the integrity of the medical literature, the BMJ now asks Dr Rita Redberg, editor of Archives of Internal Medicine, to ask Roche for the raw data that comprised the Kaiser (2003) review as described above.
It is also interesting to note that there are no clear selection criteria included in the Kaiser review. Back in 2009, we asked Drs Kaiser and Hayden why two published trials were not included in their review – the published paper gave no reasons for their omission. Specifically, these were:
Kashiwagi S, Kudoh S, Watanabe A, Yoshimura I. [Clinical efficacy and safety of the selective oral neuraminidase inhibitor oseltamivir in treating acute influenza--placebo-controlled double-blind multicenter phase III trial] [Internet]. Kansenshogaku Zasshi. 2000 Dec ;74(12):1044-1061
LI Longyun; Cai Baiqiang, Wang Mengzhao. A multicenter study of efficacy and safety of oseltamivir in treatment of naturally acquired influenza; Chin J Intern Med, December 2001, Vol 40, No. 12.
In November 2009, Dr Kaiser replied: “We always try to be as complete as possible, I do not recall specific reasons or even if I was aware of these studies; probably because the first one was in Japanese and the second one probably because our paper was accepted in October 2002.”
Dr Hayden also said: “Please forward the English translations of these articles. I do not recall the circumstances regarding access to these data. Please note that our manuscript was written in 2002 (accepted in October).”
On the 13th November 2009, Dr Kaiser also told the BMJ that ”we need to understand whether this antiviral could decrease end points like antibiotic use and potentialy related influenza complications; 10 years ago this was not available and not analyzed. We tried to collect as many results as possible in order to conduct a representative analysis, and were confronted to the usual limitations of this approach”.
Dr Hayden also said: “We conceived the study, approached Roche for access to data, worked with the statisticians on analysis, and wrote the manuscript (Dr. Kaiser wrote the first draft which I edited).”
However, the BMJ understands that the dataset for the Kaiser analysis was conceived two years before the paper was accepted for publication in 2002. Roche had conceived the idea by December 2000.
A Roche investigator's brochure from June 2009 features a table of "Summary of Secondary Illnesses Associated with Antibiotic Use (ITTI Population, Pool of Studies)". The dataset is dated 18th December 2000 and is the same as included in the review published in Archives in 2003. It included trials: “M76001, WV15670, WV15671, WV15707, WV15730, WV15812, WV15819, WV15872, WV15876, WV15978”.
There is also an interesting timeline of events. In 2000, Tamiflu received US regulatory approval from the FDA—although they rejected Roche’s claim that the drug reduced secondary complications because there was no microbial evidence. The FDA also said that there were no predetermined definitions of these illnesses – including pneumonia and bronchitis - and they were locally defined. The FDA even wrote to Roche that year to say: “You have claimed reductions in severity and incidence of secondary infections with Tamiflu that are misleading because they are not supported by substantial evidence”.
Also in 2000, Roche withdrew their application for a European licence for Tamiflu from the European Medicines Agency. Two years later, on 20th June 2002, the drug eventually received regulatory approval in Europe for secondary complications. EMA’s summary of scientific information quotes figures that later appear in the Kaiser review. It says: “The proportion of subjects who developed specified lower respiratory tract complications (mainly bronchitis) treated with antibiotics was reduced from 12.7 % (135/1063) in the placebo group to 8.6 % (116/1350) in the oseltamivir treated population (p = 0.0012).”
These are the same denominators – namely, 1063 in the ITT placebo group and 1350 in the ITT Tamiflu group - as appeared in Dr Kaiser’s review in Archives a year later. However, with the help of Roche statisticians this concluded: “among influenza-infected persons, oseltamivir use reduced the incidence of LRTCs leading to antibiotic intervention by 55% compared with placebo (4.6% vs 10.3%; P<.001)” (5).
The Kaiser (2003) review was published before the 2007 FDA Amendment Act, which mandated the public registration of clinical trials.
Given the absence of a public trial registry and the fact that eight out of the ten trials featured in review were unpublished, the BMJ would be interested to know how Drs Kaiser and Hayden went about identifying and selecting trials to include in their review. We feel publication of their approach would greatly help researchers, who are struggling to access unpublished research.
As pointed out, eight out of the ten trials in Kaiser (2003) were unpublished. This included the largest ever treatment trial of Tamiflu. The references credit this to:
Treanor JJ. Redcution in the symptoms and complications of influenza A and B with oseltamivir. Presented at: the 38th Annaul Meeting of the Infectious Disease Society of America; September 7-10, 2000; New Orleans, La. Abstract 611.
When we asked Dr Treanor why this had remained unpublished, Dr Treanor replied: “As far as I can remember, the field trial we reported in JAMA is the only large study of oseltamivir I ever participated in.”
Dr Hayden also makes the point that the conclusions of Kaiser (2003) have been supported by another meta-analysis conducted by Professors Hernan and Lipsitch at Harvard University (6). We disagree with this statement. This paper drew different conclusions to those in Kaiser (2003). It did not mention a 59% reduction in hospitalisations (p=0.2) that featured in Kaiser (2003). It was also more modest in terms of treatment effect and said that Tamiflu increased the risk of diarrhoea, nausea and vomiting by approximately 50% compared to those taking placebo. (Risk ratio (95% CI; P value for heterogeneity) was 1.46 (1.05–2.02; .06) for nausea, 1.55 (1.14–2.12; .01) for vomiting, .83 (.66–1.04; .44) for diarrhea, 1.47 (1.05–2.04; .36) for headache, and 1.02 (.72–1.44; .83) for other neuropsychiatric disorders). These side effects were not mentioned in Kaiser (2003).
We also disagree with Dr Hayden's statement that: “Perhaps of greater medical importance is the substantial body of observational studies that have found evidence that oseltamivir treatment, compared to no or very delayed antiviral therapy, reduces the risk of more serious complications and hospitalizations.”
Only randomised controlled trials can properly demonstrate efficacy, and vast sums of public money have been spent on stockpiling Tamiflu on the basis of such trials. The fact that the rationale for this vast expenditure — namely, prevention of hospitalisations; secondary complications; and stopping transmission of the virus — may not be supported by the available clinical trial evidence is of utmost concern. Once again, we urge Roche to make the raw data from all their trials available for independent scrutiny.
Investigations editor, BMJ
Editor in Chief, BMJ
1. Hayden FG, Treanor JJ, Kaiser L. Re: Search for evidence goes on. BMJ Rapid Response [Internet]. 2012 Jan 30 [cited 2012 Feb 5];Available from: http://www.bmj.com/rapid-response/2012/01/29/re-search-evidence-goes
2. Cohen D. Search for evidence goes on. BMJ. 2012 Jan 17;344(jan17 2):e458.
3. Treanor , JJ et al Efficacy and Safety of the Oral Neuraminidase Inhibitor Oseltamivir in Treating Acute Influenza JAMA. 2000;283(8):1016-1024.
4. Kaiser L, Wat C, Mills T, Mahoney P, Ward P, Hayden F. Impact of oseltamivir treatment on influenza-related lower respiratory tract complications and hospitalizations. Arch Intern Med. 2003 Jul 28;163(14):1667-72.
6. Hernán MA, Lipsitch M. Oseltamivir and Risk of Lower Respiratory Tract Complications in Patients With Flu Symptoms: A Meta-analysis of Eleven Randomized Clinical Trials. Clin Infect Dis [Internet]. 2011 Jun 15 [cited 2011 Jun 30];Available from: http://www.ncbi.nlm.nih.gov/pubmed/21677258
Competing interests: DC and FG are employees of the BMJ Group.