Recommendations to improve adverse event reporting in clinical trial publications: a joint pharmaceutical industry/journal editor perspectiveBMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i5078 (Published 03 October 2016) Cite this as: BMJ 2016;355:i5078
All rapid responses
Re: Recommendations to improve adverse event reporting in clinical trial publications: a joint pharmaceutical industry/journal editor perspective
Response to Dr. Matheson:
We thank Dr. Matheson for acknowledging the worthwhile goals of our publication and the “probity of everyone involved in the project, their careful thinking and commitment to improving journal publications,” and for stating that he has “every confidence in the integrity of the authors.” Indeed, all authors participated in developing the recommendations, vetted the examples to illustrate them, fulfilled ICMJE criteria for authorship, and submitted conflict of interest disclosures per journal requirements. Our objective was to provide a guide to more clinically relevant and informative adverse event reporting. We welcome feedback from those skilled in clinical trial design and hope that our recommendations trigger a heightened interest in reporting adverse events in a clinically meaningful manner.
Medical Publishing Insights & Practices (MPIP) was founded in 2008 by medical publication professionals in the pharmaceutical industry and members of the International Society for Medical Publication Professionals (ISMPP) to elevate trust in and the transparency and integrity of published results of industry-sponsored research. The major goal of MPIP is to encourage more effective partnerships between trial sponsors and medical journals to raise standards in medical publications and expand access to research results. For a detailed description of MPIP objectives, funding and resources, please see the MPIP website: https://www.mpip-initiative.org/about.html.
Competing interests: JAB, AB, TC, SG, CK, NL, and BM are employees of companies sponsoring MPIP, as shown by their individual affiliations. As well as the reported links with MPIP, BM is a shareholder at GlaxoSmithKline, and serves as chair of MPIP; SG owns stock and stock options in Johnson and Johnson Common Stock; MB reports salary support from the American Medical Association, outside the submitted work; CK held stocks in AstraZeneca; CL is employed by the American College of Physicians as editor in chief of Annals of Internal Medicine, a medical journal that publishes clinical trials; and LC has received consulting or speaking fees outside the submitted work from Acadia, Alexza, Alkermes, Allergan, AstraZeneca, Avanir, Boehringer Ingelheim, Bristol-Myers Squibb, Eli Lilly, Forum, Genentech, Janssen, Jazz, Lundbeck, Merck, Medivation, Mylan, Neurocrine, Novartis, Noven, Otsuka, Pfizer, Reckitt Benckiser, Reviva, Shire, Sunovion, Takeda, Teva, Valeant, and Vanda, and has stock ownership in Bristol-Myers Squibb, Eli Lilly, J and J, Merck, and Pfizer.
I will not dwell on the content of this drug industry/journal editor “joint perspective” on adverse event reporting. Its call to retire the phrase “generally well tolerated” is welcome; its emphasis on “clinically relevant” adverse events may raise concerns. But my concern is with the interests this project serves.
I acknowledge the probity of everyone involved in the project, their careful thinking and commitment to improving journal publications. The top-line goals are plainly worthwhile. But why were the authors all industry executives and journal editors? Why did the editors not insist experts in clinical trials and drug harms take part? Why not industry critics? Why, above all, should it be considered the business of scholarly journal editors not merely to talk to drug executives, but to craft exclusive “joint perspectives” with them?
To address these questions, we must consider not only this project’s top line goals, but also the bottom line – its commercial functions. It has become a common strategy in contemporary capitalism for corporations to court public trust by means of well-publicized and sincere outreach programs, dialogues, worthy projects and superficial reforms. Such practices are referred to within the corporate branding sector as “reputation management”, “stakeholder engagement” and “brand alignment”.[2-4] The Medical Publications Insights and Practices (MPIP) initiative, which instigated this project, is in my evaluation consistent with these PR trends.
MPIP was created by drug companies and marketers specializing in publications. Both profit from the drug publications trade. MPIP is sincere in its stated goals and these have value, but it also performs legitimizing functions for this trade. There could be no better means of building legitimacy than an article proclaiming unity between the drug industry and journal editors – better still, one published in a prestige journal, and it can be no surprise that the letters “BMJ” are currently up in lights on the MPIP homepage.
That might not please the BMJ; but the truth is that many publishers solicit industry patronage, whether from advertising or by leveraging publications. Sage, one of the publishers with an editor-coauthor on this project, “offers a full range of marketing solutions to help you connect with the decision makers and influencers in your field of expertise. Our sales team will work with you to develop an integrative marketing approach that delivers the maximum impact for your product.” Another, Wiley, “can develop a customized communications plan to support your promotional and publication strategy.” The drug companies, marketing agencies and publishing houses are the triumvirate who control and thrive on the trade in industrialized medical knowledge. Theirs is certainly a joint perspective.
How, then, did this “joint perspective” on adverse events come about? It was an MPIP operation, planned, bankrolled and managed by drug companies and their agents, with input from marketers, publishers and selected editors. It is not explained on what basis these particular editors were enlisted, but while they made substantial contributions at various points, the first draft was written without their participation. These are enduring arrangements: each year MPIP initiates a project, editors are recruited and a fresh joint perspective is crafted and publicized. Past output has been criticized for commercial expediency.[9,10]
My chief concerns with this project, then, are that despite meritorious top-line goals, it has an unreasonably restricted authorship, and does not adequately relate its commercial and PR dimensions. As a supporter of innovative pharmaceutical research, I encourage my industry friends to continue their drive to improve publications, but always to describe commercial functions with unsparing candour, and as conspicuously as those “BMJ” letters are chalked up on their website.
It is also notable that none of the editors serving as a coauthor on this article reports any competing interests. While I have no doubt they all made what they considered to be full and careful disclosures, this raises questions about the breadth of disclosures that should be considered appropriate in a project like this.
Consider first that many publishers seek drug industry patronage. Editors appointed or employed by such publishers should therefore, in my opinion, disclose their employment status and their employer’s interest. Dr Berlin, the courteous corresponding author, has related to me that flight and hotel expenses for editors were paid for by journal publishers. This too should in my opinion be reported.
Second, considering that this project was drug industry financed, planned, managed and drafted, with drug industry coauthors and drug industry-relevant output, my view is that drug industry payments to authors should be declared. According to Sunshine Act data, Dr Haller received US$82,859 in payments and expenses for various activities during 2013-15, the most recent years for which data are available, albeit not directly from MPIP proprietor companies. During the same period, the data indicate that Dr Citrome received US$1,313,274, the largest payments being for consultancy and promotional speaking in relation to various drugs. This total includes, by my calculation, $107,578 from Janssen, $37,473 from Takeda and $33,903 from Merck. These companies are members of MPIP, their employees contributed to the roundtable meeting and Janssen and Takeda employees are coauthors on this very paper.
I have every confidence in the integrity of the authors, but it would appear we do not share a joint perspective on what disclosures are clinically relevant.
It has not escaped my notice that another scheme which MPIP is running at the moment is called “Transparency Matters.”
Much has been said about the use of “key opinion leaders” (KOLs) by marketing. I do not direct this comment to this particular article, but editors should take care not to find themselves functioning as KOLs, if not in the promotion of drugs, then in the promotion of the commercial medical literature trade.
Richard Smith famously wrote that “medical journals are an extension of the marketing arm of pharmaceutical companies.” But if I may conclude with a jot of badinage, then it is not Smith’s words that wrestle their way into my soul, but those of the late MP and diarist, Alan Clark: “it is never the slightest use telling people who they shouldn’t go to bed with.”
1. Lineberry N, Berlin JA, Mansi B, Glasser S, Berkwits M, Klem C, Bhattacharya A, et al. Recommendations to improve adverse event reporting in clinical trial publications: a joint
pharmaceutical industry/journal editor perspective. BMJ. 2016 Oct 3;355:i5078.
2. Globescan: Reputation. http://www.globescan.com/expertise/reputation.html. Accessed March 6 2017.
3. Globescan: Stakeholder Engagement. http://www.globescan.com/capabilities/stakeholder-engagement.html. Accessed March 6 2017.
4. The Reputation Institute: Brand Alignment. https://www.reputationinstitute.com/consulting/brand-alignment. Accessed March 6 2017.
5. About Medical Publications Insights & Practices. https://www.mpip-initiative.org/about.html. Accessed March 6 2017.
6. Medical Publications Insights & Practices. https://www.mpip-initiative.org. Accessed March 6 2017.
7. Sage Publishing. Advertising & Promotion. https://us.sagepub.com/en-us/nam/advertising-and-promotion. Accessed March 5 2017.
8. Wiley Online Library. For Advertisers & Corporate Partners. http://olabout.wiley.com/WileyCDA/Section/id-390236.html. Accessed March 5 2017.
9. McHenry L, Jureidini J. On the Proposed Changes to the Credibility Gap in Industry-Supported Biomedical Research: A Critical Evaluation. Ethical Human Psychology and Psychiatry 2012; 14:156-161.
10. Matheson A. Ghostwriting: the importance of definition and its place in contemporary drug marketing. BMJ. 2016 Aug 30;354:i4578.
11. Dollars for Docs: Daniel G Haller, MD. https://projects.propublica.org/docdollars/doctors/pid/119489. Accessed March 6 2017.
12. Dollars for Docs: Leslie CItrome, MD. https://projects.propublica.org/docdollars/doctors/pid/171564. Accessed March 6 2017.
13. Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med. 2005 May;2(5):e138.
14. Clark A. Diaries: In Power, 1983-1992. London, Phoenix, 1994; entry for May 15 1990.
Competing interests: I am a supporter of innovative pharmaceutical research and have affection for, and friendships within, pharmaceutical corporations and also the marketing and publications trade, whose employees I consider to be ethical and professional. Between 1994 and 2012 most of my income came from consultancy and writing services provided to pharmaceutical corporations, either directly or via marketing agencies. In 2015 I acted as a paid expert witness on behalf of the plaintiffs in a US federal legal action against a drug company.
We thank Drs. Proctor and Schumacher for their thoughtful suggestions from November 25 regarding methods to graphically display the risk and time course of adverse events in clinical trials (Figure 1). We agree that there are various methods to compute and display the cumulative incidence and/or risk of experiencing adverse events, several of which consider appropriate competing risks (eg, death or disease progression). In our review of recent reports from oncology trials, we observed a variety of methods used, including Kaplan-Meier estimates of time to event, Kaplan-Meier estimates with censoring for predefined competing risks, and cumulative incidence curves using Fine and Gray’s method for subdistribution of competing risks. As noted by Drs. Proctor and Schumacher, the particular example we presented considers only those who (eventually) experienced the adverse event, which is useful information, but not the only information one would want to see. We elaborate below.
We agree that there will always be room for improvement in the ways to represent key data and, aligned with the mission of Medical Publishing Insights & Practices, we welcome suggestions for such improvements. The examples we selected for our publication were chosen to address the key questions practicing physicians would have for successfully using any new treatment:
• What are the most frequently observed adverse events with this treatment?
• How likely are my patients to experience these adverse events?
• Are there specific timeframes after starting treatment during which the risk of adverse events is highest or additional monitoring and follow up are needed?
While a single representative table or figure may not address all 3 questions above, the aim of our recommendations is to ensure that the data in a manuscript as a whole adequately address these aspects.
As mentioned in our publication, the recommendations and examples provided are intended as a guide for better reporting of adverse events in a clinically relevant and comprehensible manner. Therefore, we thank Drs. Proctor and Schumacher for helping us to continue the dialogue and gain further insights on how to improve representation of clinical trial data (including adverse events) within the published literature.
1. Bonifazi M, et al. Trastuzumab-related cardiotoxicity in early breast cancer: a cohort study. Oncologist. 2013;18(7):795-801.
2. Rugo HS, et al. Meta-analysis of stomatitis in clinical studies of everolimus: incidence and relationship with efficacy. Ann Oncol. 2016;27(3):519-525.
3. Wolff AC, et al. Risk of marrow neoplasms after adjuvant breast cancer therapy: the National Comprehensive Cancer Network Experience. J Clin Oncol. 2015;33(4):340-348.
Competing interests: Competing interests have not changed since publication of the original article.
Lineberry et al (1) provide recommendations to improve adverse events reporting in clinical trial publications. We agree that this topic is of high importance and especially welcome these recommendations, because most publications of clinical trials report safety data but ignore timing and duration of adverse events and report only relative frequencies instead. In situations in which the time on treatment for both treatment groups is similar, the reporting of relative frequencies can be sufficient. But especially in oncology trials time on treatment between treatment groups often differs, therefore methods which incorporate time on treatment have to be used.
These methods are described by the authors as exposure-time associations and illustrated (Fig. 1) by an example taken from a publication of Nozawa et al (2). We feel, however, that these plots presented in Figure 1 should not serve as a “best-practice” example. According to the legend, the cumulative chance of experiencing particular adverse events over time is displayed. It can easily be seen that there must be something wrong with this figure since all five “cumulative chances” approach 100% although the crude proportions are 44% for stomatitis (A), 31% for thrombocytopenia (B), 22% for anemia (C), 17% for hyperglycemia (D) and 22% for pneumonitis (E) as taken from Table 2 in Nozawa et al (2). The reason for this discrepancy is that Nozawa et al (2) plotted the cumulative distribution functions of five particular adverse events only for those patients in whom such a particular adverse event was observed. This also explains the somewhat mysteriously increasing numbers at risk that simply represent the observed number of a particular adverse event over time. Therefore the number of patients which is artificially set equal to 100% is different for different adverse events and depends on the time on treatment. Thus, these plots do not provide any information on the probability (chance) of suffering from a particular adverse event. The only thing that can be derived is whether a particular adverse event occurs early or late in time.
Alternatively we would recommend to analyse adverse events either as single or composite endpoints and to consider death or progression as competing events. The cumulative probabilities can then be calculated using the Aalen-Johansen estimator for the cumulative incidence functions for both competing events. In our opinion the graphical display of the Aalen-Johansen estimator is a more adequate method to report exposure-time associations, because additionally to the information on the occurrence of a specific adverse event it displays the probability of experiencing this specific adverse event. An application of this method to safety data was reported in Proctor et al (3).
Tanja Proctor (a), Martin Schumacher (b)
(a) Institute of Medical Biometry and Informatics, University of Heidelberg, Heidelberg, Germany
(b) Institute for Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany
1 Lineberry Neil, Berlin Jesse A, Mansi Bernadette, Glasser Susan, Berkwits Michael, Klem Christian et al. Recommendations to improve adverse event reporting in clinical trial publications: a joint pharmaceutical industry/journal editor perspective BMJ 2016; 355 :i5078
2 Nowaza M, Nonomura N, Ueda T, et al. Adverse event profile and dose modification of everolimus for advanced renal cell carcinoma in real-world Japanese clinical practice. Jpn J Clin Oncol 2013; 43:1132-8. doi:10.1093/jjco/hyt121
3 Proctor, T., and Schumacher, M. (2016) Analysing adverse events by time-to-event models: the CLEOPATRA study. Pharmaceut. Statist., 15: 306–314. doi: 10.1002/pst.1758
Competing interests: No competing interests
Adverse event reporting in surgical trials and early phase studies: the need for new and joint perspectives
We support the consensus recommendations from Lineberry et al to improve credibility and transparency in publications of industry-sponsored research on informative and clinical relevant adverse event reporting(1).
We also support a similar thorough, joint and transparent process for the development of standards and consensus for reporting of benefit and harm outcomes in surgical trials, including early phase surgical studies. Recent reviews have summarised heterogeneity of outcome reporting in many surgical areas(2-6), including in the evaluation of innovative surgical procedures(7). This limits evidence syntheses and risks outcome reporting bias. For early phase studies, the reporting of selective outcomes is likely to lead to over-optimistic assessment of new interventions and under-reporting of adverse effects. Without systematic and transparent evaluation, with details reported as recommended by Lineberry et al (e.g. numerators and denominators for all events), surgeons continue to innovate without reliable information about adverse effects. This means that the true extent of harm to patients from undergoing evolving surgical techniques is uncertain and often only becomes apparent once National Registries summarise outcomes (3,8). Such disasters could potentially be minimised by the routine measurement and reporting of adverse events in early phase surgical studies.
There is an urgent need for surgeons to work with industry, the National Office for Clinical Research Infrastructure and professional bodies to develop agreed minimal mandated sets of benefit and harm outcomes to measure within each phase of surgical innovation; from evolving through to stabilised and adopted interventions. Methodology developed with the COMET initiative (www.comet-initiative.org/) now enables the identification and categorisation of benefit and harm outcomes in early phase surgical studies and the development, via consensus methods, of minimal mandated sets of benefit and harm outcomes. The recent NIHR Biomedical Research Centre award to Bristol University includes work to develop such sets, with the potential to expedite the swift rejection of unsafe or ineffective techniques and promote the efficient development of promising innovations.
Kerry N L Avery 1, Sara T Brookes 1, Hollie Richards 1, Shelley Potter 1,2, Ashley Blom 1, Rob Hinchliffe 1,3, Jane M Blazeby 1,4 on behalf of the NIHR Biomedical Research Centre Surgical Innovation Theme 1
1 University of Bristol, Bristol, UK
2 Royal Liverpool and Broadgreen University Hospitals NHS Trust
3 North Bristol NHS Trust, UK
4 University Hospitals Bristol NHS Foundation Trust, Bristol, UK
1. Lineberry N, Berlin JA, Mansi B, Glasser S, Berkwits M, Klem C et al. Recommendations to improve adverse event reporting in clinical trial publications: a joint pharmaceutical industry/journal editor perspective. BMJ 2016;355:i5078.
2. Hopkins JC, Howes N, Chalmers K, Savovic J, Whale K, Coulman KD, Welbourn R, Whistance RN, Andrews RC, Byrne JP, Mahon D, Blazeby JM; By-Band Trial Management Group Outcome reporting in bariatric surgery: an in-depth analysis to inform the development of a core outcome set, the BARIACT Study. Obes Rev. 2015 Jan;16(1):88-106. doi: 10.1111/obr.12240. Epub 2014 Nov 30. PMID: 25442513
3. Potter S, Browning D, Savović J, Holcombe C, Blazeby JM. Systematic review and critical appraisal of the impact of acellular dermal matrix use on the outcomes of implant-based breast reconstruction. Br J Surg 2015;102:1010-25.
4. Blencowe NS, McNair AG, Davis CR, Brookes ST, Blazeby JM. Standards of outcome reporting in surgical oncology: a case study in esophageal cancer. Ann Surg Oncol 2012;19:4012-8
5. Whistance RN, Forsythe RO, McNair AG, Brookes ST, Avery KN, Pullyblank AM et al. A systematic review of outcome reporting in colorectal cancer surgery. Colorectal Dis 2013;15:e548-60.
6. Bhangu A, Singh P, Pinkney T, Blazeby JM. A detailed analysis of outcome reporting from randomised controlled trials and meta-analyses of inguinal hernia repair. Hernia 2015;19:65-75.
7. Currie A, Brigic A, Blencowe NS, Potter S, Faiz OD, Kennedy RH, Blazeby JM. Systematic review of surgical innovation reporting in laparoendoscopic colonic polyp resection. Br J Surg 2015;102:e108-16.
8. Smith AJ, Dieppe P, Vernon K, Porter M, Blom AW, National Joint Registry of England and Wales. Failure rates of stemmed metal-on-metal hip replacements: analysis of data from The National Joint Registry of England and Wales. Lancet 2012;379:1199-204.
Competing interests: No competing interests
We do not need more filters and publication guidelines but full reporting. It is unrealistic to think that a publication in a medical journal that takes up 10 pages can fairly represent harms data from a clinical study report that takes up a 1000 pages. Harms data is heterogeneous and a lot is lost in tables showing only broad categories.
The authors suggest reporting “most clinically relevant” adverse events. As it is subjective it will be susceptible to bias and it is not possible to predict which adverse event will be of interest in the future. Adding this process strictly for improving publication seems unnecessary.
The authors briefly mention that full datasets should be published on independent websites. However, this can be done in many ways and placing at the end of section called “identify and communicate most clinically relevant adverse events” does not make one confident that industry will actually adhere to this.
The authors’ goal is to improve credibility of industry sponsored trials. Instead of discouraging statistical testing and changing the existing filters industry should focus on transparency for the benefit of public health. Participants in trials risk their life and health to benefit future patients and their sacrifice can only be justified if the hidden data is shared.
Competing interests: No competing interests