Point-by-point response from Roche to BMJ questions
(Published 08 December 2009)
Cite this as: BMJ 2009;339:b5374
- James Smith, international medical leader, Tamiflu1
- on behalf of Roche
Below is a point-by-point response to the questions and allegations raised by the BMJ [questions from the BMJ are in italics].
1. The Cochrane Review
We have been closely following the efforts of the Cochrane Collaboration to update its review on: ‘Neuraminidase inhibitors for preventing and treating influenza in healthy adults. A systematic review and meta-analysis’. This update will be published in the BMJ on 8th December 2009.
As part of its investigations, the Cochrane Review sought to examine in detail Professor Laurent Kaiser’s paper, ‘Kaiser L, Wat C, Mills T, Mahoney P, Ward P, Hayden F. Impact of oseltamivir treatment on influenza-related lower respiratory tract complications and hospitalizations. Arch. Intern. Med. 2003 Jul 28;163(14):1667-1672.’ - which has been extensively cited, by Roche and public health authorities, as evidence for the effectiveness of Tamiflu in reducing hospitalisations. As publishers of the review, the BMJ has been in close contact with the Cochrane Review team. In a linked feature, we intend to report on the following matters: a. that the Cochrane Review team was unable to obtain access to the complete data sets of the 10 RCTs which the Kaiser 2003 paper meta-analysed (8 of which have not been fully published in peer-reviewed journals).
Response: In addressing the issue of access to data, we note that, according to our records, Roche was initially approached with a request for data on 9 September 2009 by Philip Carter from ITN regarding a Channel 4 news request, and not the Cochrane review team itself. We believe this is in conflict with accepted standard practice: the merits of scientific studies, and debate relating to them, are of course to be conducted primarily in a scientific or clinical forum rather than via a production company. Upon our request, ITN informed us that the person with whom they were dealing was Dr T Jefferson of the Cochrane review team. It is unclear to us why the Cochrane team and Dr Jefferson in particular, would adopt this approach, given that Dr Jefferson had been a paid ad-hoc consultant to Roche between 1997-–1999 working on influenza and oseltamivir. During that time he worked closely with Roche experts, many of whom are still in the company. Therefore, Dr Jefferson should have had no issue with reaching out to the experts directly to discuss his requirements. We are further concerned that this approach appears to have been supported by the BMJ.
As you will note from the replies below, Roche is, nevertheless, very happy to have its data reviewed by appropriate authorities or individuals, and has never concealed (or had the intention to conceal) any pertinent data. Indeed, as far as we can ascertain, it was Roche who initiated contact with Dr Jefferson (11 September 2009) to determine exactly the information he was requesting.
From this contact, Roche learned that Dr Jefferson wished to receive the full study data. Over the next weeks, Roche offered to provide the data under a confidentiality agreement, as is commonplace within the scientific community to ensure the responsible use of data. Roche has, for example given full access to its Tamiflu clinical database of patient-level information to a Medical Research Council epidemiology research unit following such a confidentiality agreement. Dr Jefferson was unwilling to enter into the agreement with Roche.
Three weeks later and despite this refusal, Roche sought to provide Dr Jefferson with the key clinical data in the form of the data tables concerning influenza-related complications and hospitalizations, to allow him to address his questions directly.
As you may be aware, Roche was one of the first companies in the industry to launch an internet site for publishing study protocols and results, in 2005. When the website (roche-trials.com) was created, Roche decided to publish data from 2005 onwards. In view of the exceptional interest in Tamiflu and its key role in the pandemic, Roche has now disclosed (7 December 2009) on roche-trials.com the study summaries (including key data) relating to the Kaiser manuscript. The corresponding full study reports will also be made available on a password-protected site within the coming days to physicians and scientists undertaking legitimate analyses.
b. that it is a matter of concern that 8 out of 10 treatment trials are unpublished and therefore unverifiable by the general medical community.
Response: Today it is standard practice for Roche to publish all its clinical trial data, but this was not standard policy within Roche or elsewhere within the industry 7–10 years ago. At the time, it was considered that the studies that were published (2 abstracts and 2 full manuscripts) reflected accurately the benefits of the drug, and that the additional studies provided little new information and would therefore be unlikely to be accepted for publication by most reputable journals. However, the pooled analysis of the studies, based on greater numbers of patients, was considered to be of additional value to the single-study publications, and was thus undertaken and published as the Kaiser manuscript.
We would further add that all of these study reports were submitted to health authorities including the US FDA and EMEA for their review, and that the dates of the submissions were provided to Dr Jefferson in response to his request for data. As outlined above, to ensure transparency of process and results, Roche has now disclosed on roche-trials.com the study summaries (including key data). The full study reports will also be made available on a password-protected site within the coming days to physicians and scientists undertaking legitimate analyses.
c. that requests to Professors Kaiser and Hayden, as lead and corresponding authors, on the Kaiser 2003 paper and to the lead authors of 4 published trials: Professor J Treanor [Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, Singh S, Kinnersley N, Ward P, Mills RG. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza: a randomized controlled trial. US Oral Neuraminidase Study Group. JAMA. 2000 Feb 23;283(8):1016-1024]; Professor K Nicholson [Nicholson KG, Aoki FY, Osterhaus AD, Trottier S, Carewicz O, Mercier CH, Rode A, Kinnersley N, Ward P. Efficacy and safety of oseltamivir in treatment of acute influenza: a randomised controlled trial. Neuraminidase Inhibitor Flu Treatment Investigator Group. Lancet. 2000 May 27;355(9218):1845-1850]; Professor Li [Li L, Cai B, Wang M, Zhu Y. A double-blind, randomized, placebo-controlled multicenter study of oseltamivir phosphate for treatment of influenza infection in China [Internet]. Chin. Med. J. 2003 Jan ;116(1):44-48]; and Professor Kashiwagi [Kashiwagi S, Kudoh S, Watanabe A, Yoshimura I. [Clinical efficacy and safety of the selective oral neuraminidase inhibitor oseltamivir in treating acute influenza--placebo-controlled double-blind multicenter phase III trial] [Internet]. Kansenshogaku Zasshi. 2000 Dec ;74(12):1044-1061] for the relevant data resulted in inadequate responses or no response at all. Those that did respond referred the Cochrane reviewers to Roche, with authors stating they no longer had the data in their possession.
Response: We remind the BMJ that a number of the independent investigators have confirmed their roles in authorship of publication, study involvement and data analysis, and have already provided information to which they continue to have access.
Roche does have access to the relevant information, and has offered to resubmit it to the investigators of those studies should they require it, though as stated above Roche intends to make these data available to the scientific community.
d. concerning the Kaiser 2003 paper, in respect of which paper its lead author Professor Kaiser has confirmed to the BMJ that Roche performed the statistical analysis and data base management, that the following two RCTs which were apparently relevant to the research were not included in the meta-analysis: Kashiwagi S, Kudoh S, Watanabe A, Yoshimura I. [Clinical efficacy and safety of the selective oral neuraminidase inhibitor oseltamivir in treating acute influenza--placebo-controlled double-blind multicenter phase III trial] [Internet]. Kansenshogaku Zasshi. 2000 Dec ;74(12):1044-1061
LI Longyun; Cai Baiqiang, Wang Mengzhao. A multicenter study of efficacy and safety of oseltamivir in treatment of naturally acquired influenza;; Chin J Intern Med, December 2001, Vol 40, No. 12.
Response: Roche was fully aware of these papers, which were based on Roche-supported studies. The databases for these papers were written in Japanese and Chinese, respectively, and at that time it was not possible to translate and re-verify the contents item by item within the timeline constraints for the analysis and publication.
The English-language translations of the full Chinese paper and the Japanese abstract (full paper currently in translation) have now been obtained, and it is important to note that both reports support the conclusions of previous studies. We will contact the journals concerned to determine their willingness to make full English-language translations available.
2. Observational data
Together with Channel 4 News the BMJ has commissioned Professor Nick Freemantle, Professor of Clinical Epidemiology & Biostatistics at the University of Birmingham, to review the observational data studies which Roche supplied to the Cochrane reviewers, with regard to study design and potential for confounding factors to exert a non-trivial influence on outcomes.
The BMJ has interviewed Professor Freemantle and intends to include in its report:
a. Professor Freemantle’s conclusions that, whilst such observational data suggest that oseltamivir may lead to reductions in the risk of pneumonia among otherwise healthy individuals who contract influenza, the absolute benefit is “small” and side effects and safety should also be considered.
Response: We would emphasise that the benefits of oseltamivir have been shown in randomised controlled clinical trials, not simply through observational trials. Professor Freemantle’s point about benefits may be the case in seasonal influenza where complications are relatively infrequent, but can change with increasing pathogenicity of either avian or pandemic viruses.
We agree that the benefits should always be considered together with side-effects of any drug. Concerning side-effects of Tamiflu, the drug has been used by 68 million patients worldwide over the last 10 years, and safety and tolerability profiles are therefore well established, through both clinical studies and spontaneous adverse event reporting from market use.
b. Professor Freemantle’s comments to the effect that he believes that the treatment effects of Tamiflu are small and that he has seen little evidence to support the widespread use of Tamiflu in the otherwise healthy population who are developing influenza like illness.
Response: Professor Freemantle’s position is broadly in agreement with that of the World Health Organization (WHO) for the clinical management of pandemic (H1N1) 2009 infection, namely, that: “Patients not considered to be at higher risk of developing severe or complicated illness and who have uncomplicated illness due to confirmed or strongly suspected influenza virus infection need not be treated with antivirals.”
Roche agrees with this WHO position, but also notes the WHO’s observations that around one third of patients with very severe illness admitted to intensive care units were previously healthy persons.
Furthermore, Roche concurs with the WHO guidelines, which highlight the importance of treating individuals in high-risk groups – which can include:
Infants and young children, in particular <2 years
Persons of any age with chronic pulmonary disease (e.g. asthma, COPD)3
Persons of any age with chronic cardiac disease (e.g. congestive cardiac failure)
Persons with metabolic disorders (e.g. diabetes)
Persons with chronic renal disease, chronic hepatic disease, certain neurological conditions (including neuromuscular, neurocognitive, and seizure disorders), hemoglobinopathies, or immunosuppression, whether due to primary immunosuppressive conditions, such as HIV infection, or secondary conditions, such as immunosuppressive medication or malignancy
Children receiving chronic aspirin therapy
Persons aged 65 years and older
We would also like to highlight findings from a recent US study of patients hospitalised with 2009 pandemic influenza A (H1N1) virus. The study showed that the only intervention that made a statistically significant difference in helping patients to recover was an antiviral drug, and that 90% of those who had taken antivirals in the study had taken Tamiflu (Jain et al. N Engl J Med 2009;361:1991–3).
The BMJ intends to report that the Cochrane reviewers also had concerns about the exact roles and identities of individuals involved in the trials for Tamiflu and in subsequent published papers. These concerns arise from the fact there are apparent inconsistencies in the personnel listed in some of the data sets Roche provided to the Cochrane reviewers when compared with the personnel listed in either the corresponding published papers, or with trial reports provided by Roche to the UK public health body NICE, which we have had sight of. This poses serious questions over the precise terms of authorship of the published literature. For example: a. The individual responsible for the WV15670 trial in data provided by Roche to the Cochrane reviewers is named as “J. Dorkings”, yet in the published trial report, [Nicholson KG, Aoki FY, Osterhaus AD, Trottier S, Carewicz O, Mercier CH, Rode A, Kinnersley N, Ward P. Efficacy and safety of Oseltamivir in treatment of acute influenza: a randomised controlled trial. Neuraminidase Inhibitor Flu Treatment Investigator Group. Lancet. 2000 May 27;355(9218):1845-1850)], there is no mention of the name “J. Dorkings”. We understand that J Dorkings is an employee of Roche.
Response: J. Dorkings is an employee of Roche and was responsible for compiling a pharmacology report from a substudy of study WV15670.
The cited Nicholson paper was based on the clinical and safety results and not pharmacology, and therefore it would have been inappropriate to name J Dorkings as an author.
In this context the BMJ also intends to report that in Roche’s submissions to NICE, another named individual is attributed a role in trial WV15670 who does not feature in the authorship list included in the relevant published paper. The individual in question is “R Rutherford”.
Response: R. Rutherford was an employee in the Clinical and Regulatory Documentation group at Roche, responsible solely for compiling sections of the report written by relevant Roche experts (and not inception, design, analysis or interpretation), therefore again did not meet the eligibility criteria for authorship.
Furthermore, in the same set of submissions to NICE, the personnel listed as being responsible for clinical and statistical analyses on WV15670 are: “P. Ward, Clinical Science Leader”, “L. Struthers Statistician” and “N. Kinnersley, Statistician”. But, as above, on the published RCT other names appear as authors: “K. G. Nicholson, F. Y. Aoki, A. D. M. E. Osterhaus, S. Trottier, O. Carewicz, C. H. Mercier and A. Rode”.
Response: P. Ward, N. Kinnersley were the internal experts closely involved in the study design and clinical and statistical analysis and are therefore the named Roche experts in the author list. By naming them we make it clear that Roche staff were involved in the development of this publication. (Roche financial support is also explicitly declared in the acknowledgements of the paper.)
Furthermore, the publication acknowledges on page 1849 that a writing committee was formed, consisting of both internal Roche employees (including P Ward and N Kinnersley) and external authors from among the group of study investigators.
The manuscript further acknowledges all the investigators who participated in this study. L. Struthers was the program statistician and would have been in overall responsibility for the statistical evaluations of all oseltamivir clinical trials, and would have signed off all documents that were submitted to health authorities. N. Kinnersley was the study statistician, responsible for the individual treatment studies.
b. The published paper, ‘Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, Singh S, Kinnersley N, Ward P, Mills RG. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza: a randomized controlled trial. US Oral Neuraminidase Study Group. JAMA. 2000 Feb 23;283(8):1016-1024”’, makes no mention of the individuals “Struthers L” or “J. Dorkings”. However, both these individuals appear in Roche data provided to the Cochrane reviewers—Struthers as the statistician for the trial and Dorkings as lead investigator.
Response: The published paper by Treanor et al. contains data from another clinical trial: WV 15671. J Dorkings was again the Clinical Pharmacologist for a pharmacology substudy only (which was reported separately), and was therefore mentioned in the pharmacology report. L. Struthers was the program statistician in overall responsibility for the statistical evaluations of all oseltamivir clinical trials, and signed off documents submitted to health authorities. N. Kinnersley was the study statistician, responsible for the individual treatment studies.
The BMJ also intends to report that in the submissions to NICE, another named individual is attributed a role in trial WV15671 who does not feature in the authorship list included in the relevant published paper. This individual in question in this case is “LN Pierre”.
Response: L. Pierre was an employee in the Clinical and Regulatory Documentation group at Roche responsible solely for compiling sections of the report written by relevant Roche experts (and not inception, design, analysis or interpretation), therefore again not meeting the eligibility criteria for authorship.
Moreover, in the same set of submissions to NICE, the personnel responsible for clinical and statistical analyses on WV15671 are listed as: P. Ward, Clinical Science Leader; R. Mills, Clinical Director; L. Struthers, Statistician; N. Kinnersley, Statistician. However, on the published RCT other names appear as authors: Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, and Singh S.
Response: P. Ward, R. Mills, L. Struthers and N. Kinnersley were the internal experts responsible for the study design, and the clinical and statistical analysis of the study data included in the clinical study report submitted to health authorities. The other authors on this publication represent a subset of the investigator group who assumed responsibility for the content of the publication. Again, on pages 1023 and 1024, all the contributors are acknowledged. All who were responsible for and signed off the research are mentioned in the Acknowledgements.
In this context, the BMJ also intends to report: a. that an individual, working for a medical communications company called Adis, was involved in the preparation of the above papers, without that person’s role being disclosed (i.e. a ‘ghost-writer’);
Response: Roche confirms that medical writers were used to help draft some of the above papers. This is neither unusual nor secretive, and is common practice in the scientific community. At the time of writing and submission (2002), it was not standard practice for professional medical writers to be named on manuscripts.
What has changed since the date of preparation of those manuscripts is the introduction in 2003 of the Good Publication Practice guidelines for Pharmaceutical Companies (GPP), since when it has become accepted practice to individually name professional medical writers. When these papers were written, this was not the case, although Roche support of the study overall was declared in the Acknowledgements section of the paper. Since the introduction of GPP, Roche fully complies with the guidelines, and the recently published update, GPP2.
b. that, according to one of the medical writers from Adis who worked on the Tamiflu account, Roche’s marketing department involved itself in the preparation of journal articles including Nicholson (2000) and Treanor (2000). Roche’s marketing department, according to the BMJ’s source, obliged writers of scientific papers to include in the papers a series of “key messages” that included emphasising the importance of influenza and the role of Tamiflu in treating it; and
Response: Although common at the time, the term “key messages” is no longer used in the context of publications as it has been frequently misinterpreted. We are not clear if there is a specific inclusion in the manuscript that is considered inappropriate, imbalanced or inaccurate; if so, this has not been made clear to us. Roche also refutes that it obliged writers to include any wording or component of the content: content of any paper involving a medical writer would be developed in conjunction with the authors, and would be reviewed and approved by them before submission, and under their control at all times. We have records verifying that the authors had access to data and provided review and comments on the manuscripts.
c. comments provided to us by Dr Ike Iheanacho, Editor of the Drug and Therapeutics Bulletin, who stresses the fundamental importance of transparency and clarity i.e. that the scientific community must know exactly who is responsible for trials that have been carried out. We also note the requirements of the International Committee of Medical Journal Editors (“ICMJE”) on authorship when submitting manuscripts to Biomedical Journals.
We agree with and support the transparency of involvement; you will note the apparent contradictions you cited above were all misunderstandings of role, some based on documentation submitted to NICE rather than publicly published. We also agree on the role and criteria needed for authorship, based on those presented by ICJME: as mentioned above, those named as authors made substantial contributions to the study, were involved in manuscript drafting or review, and approved the submission.
3. Inconsistencies between company reports and published papers The BMJ also intends to make reference to certain apparent inconsistencies between information provided to NICE in Roche’s internal company reports of certain Tamiflu-related trials and information disclosed in the published papers germane to these trials: a. In Roche’s submissions to NICE dated May 2000, the volume outlining WV15670, it was stated “PUBLICATION (REFERENCE) None”. However, it appears to the BMJ that the relevant published paper, Nicholson (2000), would have been ‘in press’, pending publication, at this time, given that the paper was published in late May 2000. The NICE submission omitted to record this fact. Similarly, in the volume outlining WV15671, also submitted to NICE in May 2000, it is stated: “PUBLICATION (REFERENCE) None”. However, the equivalent paper, Treanor (2000), had by then already been published in February 2000.
Response: The original submission to NICE was prepared during 1999, before the submission of the articles mentioned above. This submission was subsequently withdrawn in 2000, following the decision to withdraw the European license application. A new submission was then made in 2002, when each of the above references was cited in the application. The NICE evaluation that accompanied the European approval of oseltamivir was therefore made in full recognition of the available information.
b. Roche’s company reports submitted to NICE also appear to reveal important information about the study design of the Treanor (2000) and Nicholson (2000) papers — information bearing upon the generalisability and the reader’s general understanding of the conclusions stated in those papers—which was not included in the methods sections of the published papers. The information that the BMJ has in mind in this regard includes the following: The Roche company reports for trials WV15670 and WV15671 submitted to NICE comprise the following statement: “Centers were activated to recruit subjects during an influenza outbreak in the locality, detected using standardized surveillance techniques.” This information was not included in the relevant published paper. The BMJ has discussed this omission with Dr Tom Jefferson. Dr Jefferson’s response was that, if this information had been included in the published paper, this would have affected how the paper was applied in everyday practice. In other words, according to Dr Jefferson, the omission was a significant one. Why, on Roche’s account, was this information not included in the published paper?
Response: We would draw your attention to page 1668 of the Kaiser publication under “Patients and Drug Administration”, where the environmental information is in fact included: “All patients were enrolled during periods when influenza virus was documented to be circulating in their communities.” Before these studies were started, scientific advice was obtained from the health authorities on key aspects of the study design and conduct, including centre activation and surveillance techniques, and it was agreed with them that this approach was acceptable to increase the probability that patients entering the studies were indeed infected with influenza.
Similarly, in the Nicholson (2000) publication, it was stated that recruitment occurred during January to March, 2008”, i.e. during the influenza season.
To address Dr Jefferson’s comment that timing would affect how the paper was applied in practice, we refer to the European Summary of Product Characteristics, which dictates recommendations for everyday use: “In adults and children one year of age or older who present with symptoms typical of influenza, when influenza virus is circulating in the community.” This comment is therefore fundamentally addressed through the registered indication of the drug, indicating that the influenza environment was both declared and appropriate.
c. Roche’s company documents related to WV15670 which Roche submitted to NICE included the following statement: “The study was originally designed to enrol 750 subjects, however, due to the low incidence of influenza in the Northern Hemisphere the actual number of subjects entered into the study to receive either active drug (75mg/150mg b.i.d. Ro 64-0796 (GS 4104)) or placebo totalled 726.” Patients, it is stated, were recruited to the trial on the basis that they displayed the clinical symptoms of influenza—termed collectively ‘influenza like illness’ (“ILI”). At a later point in the same document, it is stated that 64% to 69% participants in all 3 arms of the trial had ILI caused by influenza. This, according to Dr Jefferson, is an extraordinarily high rate for ‘ILI caused by influenza’. The rate of ILI due to seasonal influenza, the BMJ understands, is usually around 10-15%.
Given the report says that the incidence of influenza was low, it is not clear how Roche accounts for the extraordinarily high rate of ILI caused by influenza in the trial arms. Could you please explain?
Response: During the design of its studies, Roche worked with both health authorities and investigators to agree a list of symptoms to be used to define ILI. Laboratory analysis of samples from patients with these symptoms meeting this agreed definition confirmed that 64%–69% had influenza. It is not clear how Dr Jefferson reached the figure of 10%–15%, and we would welcome clarification of this. Other publications also cite significantly higher percentages, for example, 71% of all participants had laboratory-confirmed influenza in a study of the efficacy and safety of zanamivir that also used surveillance to confirm that influenza was circulating in the community (Silagy C et al., Lancet 1998; 352:1877–81).
Another concern is that none of the matters referred to above—importantly the information about the low levels of influenza prevalent at the relevant time—are mentioned in the published paper, Nicholson (2000). Again, the BMJ has discussed these omissions with Dr Jefferson. Similarly, Dr Jefferson’s response was that, if this information had been included in the published paper, it would have affected how the paper was interpreted by the medical community. In particular, its publication would have allowed the medical community to evaluate the effectiveness - as opposed to the efficacy - of Tamiflu. Why, on Roche’s account, was this information not included in the published paper?
Response: As we have explained above, the influenza environment was in fact mentioned in both papers, although not numerically so. Furthermore, both manuscripts present information for the total treated population (ITT) and the population infected with laboratory-confirmed influenza (ITTI). It was agreed with health authorities that the analyses for efficacy would be performed on the laboratory-confirmed infected population. Both manuscripts demonstrate that there was a meaningful and significant treatment effect in the total population compared with placebo.
We also note that the BMJ and Channel 4 had a concern regarding the use of the Fisher Exact test in the methodology of the Kaiser manuscript. We have now repeated and extended these analyses using additional statistical methodology that controls for study in a pooled analysis; this further supports the conclusions reached and the suitability of the process. Using the alternative of the Cochran-Mantel-Haenszel test, controlling for study, a significance value of p<0.0001 is reached – more significant than the Fisher’s test value in the published manuscript. Furthermore, both the Breslow Day test and Zelen’s Exact test give p>0.1, showing that no heterogeneity was detected and confirming the appropriateness of combining the studies in a pooled analysis.
Cite this as:BMJ 2009;339:b5374