Pharmaceutical companies’ policies on access to trial data, results, and methods: audit study
BMJ 2017; 358 doi: https://doi.org/10.1136/bmj.j3334 (Published 26 July 2017) Cite this as: BMJ 2017;358:j3334- Ben Goldacre, senior clinical research fellow1,
- Síle Lane, director of policy2,
- Kamal R Mahtani, senior clinical research fellow1,
- Carl Heneghan, professor1,
- Igho Onakpoya, research fellow1,
- Ian Bushfield, policy officer2,
- Liam Smeeth, professor3
- 1Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford OX2 6GG, UK
- 2Sense about Science, London, UK
- 3London School of Hygiene and Tropical Medicine, London, UK
- Correspondence to: B Goldacre ben.goldacre{at}phc.ox.ac.uk
- Accepted 3 July 2017
Abstract
Objectives To identify the policies of major pharmaceutical companies on transparency of trials, to extract structured data detailing each companies’ commitments, and to assess concordance with ethical and professional guidance.
Design Structured audit.
Setting Pharmaceutical companies, worldwide.
Participants 42 pharmaceutical companies.
Main outcome measures Companies’ commitments on sharing summary results, clinical study reports (CSRs), individual patient data (IPD), and trial registration, for prospective and retrospective trials.
Results Policies were highly variable. Of 23 companies eligible from the top 25 companies by revenue, 21 (91%) committed to register all trials and 22 (96%) committed to share summary results; however, policies commonly lacked timelines for disclosure, and trials on unlicensed medicines and off-label uses were only included in six (26%). 17 companies (74%) committed to share the summary results of past trials. The median start date for this commitment was 2005. 22 companies (96%) had a policy on sharing CSRs, mostly on request: two committed to share only synopses and only two policies included unlicensed treatments. 22 companies (96%) had a policy to share IPD; 14 included phase IV trials (one included trials on unlicensed medicines and off-label uses). Policies in the exploratory group of smaller companies made fewer transparency commitments. Two companies fell short of industry body commitments on registration, three on summary results. Examples of contradictory and ambiguous language were documented and summarised by theme. 23/42 companies (55%) responded to feedback; 7/1806 scored policy elements were revised in light of feedback from companies (0.4%). Several companies committed to changing policy; some made changes immediately.
Conclusions The commitments made by companies to transparency of trials were highly variable. Other than journal submission for all trials within 12 months, all elements of best practice were met by at least one company, showing that these commitments are realistic targets.
Introduction
The methods and results of completed clinical trials are routinely left unpublished.1 This is a longstanding structural problem that impacts negatively on patient care.2 3 Anecdotally there is a wide range of variation in policies and actions on trial transparency between different companies. For example, GlaxoSmithKline has publicly committed to share clinical study reports (CSRs) for all clinical trials back to 2000,4 and it has set up a unit within the company to deliver this.5 In contrast, AbbVie and Intermune sued the European Medicines Agency in a bid to prevent the regulator from disclosing the equivalent documents.6
Audit is a simple tool that is widely used throughout medicine to help improve standards by establishing a reference standard, against which performance can be measured.7 Through audit, those performing badly can be targeted for action to improve standards, and those performing to the highest standards can be identified, so that others can learn from their best practice. Audit data can be used by stakeholders such as regulators, patient groups, professional bodies, ethical investors, and healthcare workers to advocate for improvements in poorly performing companies and to help improve standards—for example, by informing individual consumer decisions and policy activity,8 or by informing decisions made by ethical investors, as exemplified by the Access to Medicines Index.9 There have been calls for audits of access to trial results for performance monitoring and comparison,8 with several recent examples including trials from individual research centres,10 11 12 all drugs approved in one year,13 and all trials approved by individual ethics committees.14 No attempts have, however, been made to systematically compare funder or sponsor policies on transparency.
We identified pharmaceutical companies’ policies on access to information about clinical trials and extracted structured data characterising their commitments to register clinical trials, share summary results, share CSRs, and give access to trial data for current and past trials (box 1); and we assessed concordance with ethical and professional guidance.
Box 1: Types of information about a clinical trial
-
Registration is the most basic level of transparency: an entry on a publicly accessible registry to note that the trial exists, with some core information on features such as the intervention and the patient population.15 Registration does not guarantee disclosure of results, but it can be used by researchers to identify completed trials as a step towards establishing whether they have subsequently disclosed their results.
-
Methods and summary results of clinical trials can be reported in an academic paper, or as structured tables of information on registers such as clinicaltrials.gov.16 Both methods have strengths and weaknesses.17 18 19 20 21 However, academic publication remains the most common route for the communication of trial results.
-
Clinical study reports (CSRs) are large documents, sometimes thousands of pages long, which are generated for regulatory purposes and follow a standard format set out under international guidance.22 They are routinely created for industry trials, and less well known in the academic community, but contain a wealth of detail on methods and results that is often missing from other sources23: one recent study estimates that CSRs contain twice as much information on benefits and harms as academic papers on trials.24 From 2010 the European Medicines Agency began releasing CSRs on request, after a European ombudsman ruling of maladministration against the agency for withholding such information.25
-
Individual patient data (IPD) is the raw data collected during a clinical trial, with detailed information on each individual participant. As such it presents important opportunities for research—for example, by allowing third parties to verify trialists’ initial analyses; permitting meta-analysis of pooled IPD for more accurate point estimates of benefits; giving greater power for subgroup analyses; and allowing new hypotheses to be explored in existing data, including on abandoned products and treatments.26 27 28 However, it also presents a risk of re-identification of pseudonymised participants. Because of this, IPD is not generally posted in public but shared through various controlled access mechanisms, as with other forms of rich electronic health record data used by epidemiologists.
Methods
Search strategy
We set out to include 50 companies: the top 25 pharmaceutical companies by global sales29 and an arbitrary selection of smaller companies for exploratory analysis of smaller firms’ policies. Baxter was excluded as it no longer makes pharmaceutical products and Teva was excluded as it is principally a generics company; during the audit period six smaller companies ceased to exist, largely through merger, leaving 42 companies. We searched Google for company policies and statements on clinical trial transparency using the key terms “company name” “clinical trial transparency” and “company name” “clinical trials” and by navigating through company websites for formal standalone transparency policies. We also searched for policies on clinical trial transparency from the European Federation of Pharmaceutical Industries and Associations (EFPIA) and Pharmaceutical Research and Manufacturers of America (PhRMA). We saved archive copies of all company website pages containing transparency commitments, downloaded archive copies of all standalone documents such as PDFs, and downloaded policy documents from EFPIA and PhRMA, as they stood at 17 April 2016.
Defining the reference standard for transparency commitments
In line with best practice for audits, we established the reference standard for a transparency policy and developed a data structure to reflect this standard. Our reference standard for transparency was that all trials should be registered as per International Committee of Medical Journal Editors (ICJME) requirements,30 31 World Health Organisation guidance,32 33 and legislative requirements34; with methods and results reported in summary form within 12 months of trial completion through online results reporting or other publication, as required under WHO guidance,1 EU legislation,35 36 and Food and Drug Administration Amendments Act (FDAAA) of 2007; that CSRs should be made publicly available if they have been created, in accordance with current EU legislation36 and various calls from civic society and academia,23 37 and that IPD should be available on request in some form to researchers.38 39 We then operationalised these broad commitments into structured questions across the four domains of registration, methods and results sharing, CSRs, and IPD, assessing the policy commitments on each domain in detail (see web appendix 1). Prospective commitments and retrospective commitments were coded separately. Because some companies’ retrospective commitments only applied to recent trials, whereas others went back several decades, we extracted the start dates for retrospective commitments into our coding sheet. We also assessed whether certain categories of trial, such as phase IV trials conducted after approval of a new product, or trials of unlicensed medicines and off-label uses, were included under each policy.
Data extraction
Five experienced researchers (BG, CH, KRM, IO, and SL) with a background in clinical trials, transparency, or research integrity (or a combination of specialties) independently extracted the data from retrieved documentation and websites into a data extraction sheet reflecting the questions in web appendix 1. At least three researchers independently extracted data, and then met to agree the final coding by consensus. In some circumstances it was not possible to code answers as “yes” or “no”: these were coded as “unclear.” We attempted to minimise use of this code and achieve consensus where possible. Additionally, we collected examples of ambiguous, contradictory, or problematic commitments during coding, and grouped these by theme.
Engagement with companies
In 2015 before commencing this study we wrote to each companies’ representatives inviting them to meet us for an hour. This was to explain our project, allow for feedback, and ensure that it was understood we would be collecting data on policies and publishing our findings. In June 2016 we sent the companies the full set of data extracted for their company’s policy and invited responses setting out any disagreements on any element, by making reference to the text of their publicly accessible policy as it stood at 17 April 2016. The data sent did not indicate how the company compared with other companies. We sent emails to the chief executive officer, medical director, and email accounts of other individuals in relevant roles who had previously responded to us on related queries about transparency. All responses were read, themes and disagreements extracted and reviewed by at least two members of the team (BG and SL), and changes made where appropriate.
Data synthesis and analysis
We generated descriptive statistics summarising the proportion of companies making key commitments and the extent to which commitments applied retrospectively. For companies that were members of an industry body, we assessed whether their policy was consistent with the minimum commitments made by four pharmaceutical industry bodies: EFPIA, PhRMA, International Federation of Pharmaceutical Manufacturers and Associations (IFPMA), and Japan Pharmaceutical Manufacturers Association (JPMA). All raw underlying data are shared online,40 permitting others to critically review assessments or create composite measures of overall transparency commitments to compare companies.
Patient involvement
The development of the overall research question and outcome measures was informed by the AllTrials campaign’s extensive engagement with signatories and supporters, including patient groups. Patients were not formally involved in developing the study design.
Results
Overall, 42 companies were assessed: 22 were based in the European Union, 13 in the USA, six in Japan, and one in Canada. Forty companies (95%) had a publicly accessible policy, and in total we reviewed 527 pages of policy documentation. Table 1⇓ shows the proportion of companies that met each of the transparency criteria. In total, 21 (91%) of the 23 top companies by revenue had a commitment to register all trials, 15 (65%) described their registration policy covering past trials, and two (9%) conducted an audit of compliance. Twenty two (96%) companies made a commitment to make all summary results available, 17 (74%) committed to share the summary results of past trials; however, policies commonly did not include timelines for disclosure, and only six (26%) included trials on unlicensed medicines and off-label uses. Twenty two companies (96%) had a policy on clinical study reports (CSRs), of whom 21 offered some form of sharing (17 on request and two sharing synopses); two included trials on unlicensed medicines and off-label uses. Twenty two companies (96%) had a policy to share individual patient data (IPD). One included unlicensed medicines and off-label uses and 14 included phase IV trials. Table 1⇓ shows that the exploratory group of smaller companies made fewer transparency commitments.
Table 2⇓ lists all the companies and policy documents coded, and summarises whether each committed to the standards described. The full extracted data on every company’s detailed commitments are available online.40 The median start date for retrospective policies on the reporting of both registration and summary results was 2005. The median start date for sharing of both CSRs and IPD was 2012. Table 3⇓ shows the range of the start dates for policy commitments.
Problematic, inaccurate, and contradictory language
Table 4⇓ gives examples of problematic, inaccurate, and contradictory language in policy documents, grouped by theme (see web appendix 2 for a longer list). Several policies used the word “all” problematically: it was stated or implied in one place that a commitment applied to “all” trials but then a caveat was applied elsewhere in the documentation. Several policies used ambiguous language. For example, Merck Serono stated, as a commitment: “All Merck Serono clinical trials in patients will be considered [our emphasis] for publication in the scientific literature, regardless of outcome.” Similarly, several policies included poorly defined caveats about which trials were covered by the commitments. For example, Purdue “has committed to publish in a publicly available database the results of many of its clinical trials”; and Sanofi commits to post results for “phase I to IV clinical trials conducted in patients, and for some [our emphasis] vaccines trials conducted in healthy subjects.” Some companies made commitments so broad that they were either improbable or contradicted by other parts of the policy. For example, Lundbeck lists a series of exclusions to its transparency policy, setting out those trials of which they will not report results, but also said it adheres to the Declaration of Helsinki, which requires all results to be made publicly available. Further anomalies were identified. Some companies committed to share results on platforms that do not appear to exist. Several companies’ policies contained clauses implying concern that sharing summary results within a 12 month timeline required by various regulations would compromise academic journal publication.
We did, however, also find examples of good policies and exemplary clear language. For example, Merck explicitly states that it applies “the same ethical standards to clinical trials in all countries including the developing world”; whereas Bristol-Myers Squibb makes an uncommon explicit commitment to submitting all phase IV trials for journal publication (“We commit to submitting all phase III and IV clinical trials regardless of outcome to peer reviewed journals for publication”). Conversely, Novartis explicitly excludes phase IV trials from CSR and IPD sharing: although less than ideal, the policy is clear and unambiguous on this issue.
Coding challenges
We encountered various coding challenges. Some companies had different policies for trials in different territories. Because clinical trials research is a mobile global enterprise we coded according to the elements of the policy that applied globally. Some companies committed to adhere to the law on sharing summary results (eg, Pfizer: “after the completion of those studies, we provide results on clinicaltrials.gov and other registries, in accordance with local regulations and guidelines”). However, these statements are difficult to interpret, since the regulations and guidelines themselves are often poorly specified and implemented: for example, the rules implementing the FDA Amendment Act 2007 were still not published when this audit was completed, and they do not apply to all trials. We therefore scored companies for a commitment only if it was explicitly stated what they would share, rather than alluding to compliance with regulations. Some companies stated they shared information on trials for treatments that were either approved or where the research programme was terminated. We coded these as not committing to share results of trials on unapproved medicines (or off-label uses of approved medicines) because the process of termination is discretionary and may not happen within a consistent timeframe. Related to this, many companies stated they will submit results of previous trials to clinicaltrials.gov but gave no timeline: again, this meant results might never appear and yet not formally breach the company’s policy commitments. Undated documents often made it impossible to assess how far back policies go; for these companies we could not give a policy start date. Where policy start dates were given, it was challenging to make dates comparable between companies, as some stated “trials started after” a given date, some “trials completed after” a given date, some “drugs approved after” a given date, and some simply gave a date without further specification. We normalised dates using the method described in web appendix 1.
Company feedback
When sent details of our extracted summary of their commitments, 23/42 companies (55%) replied. These replies were highly variable in length (mean 12 pages, range 1-39) and content. Two companies (Pfizer and GlaxoSmithKline) provided publicly accessible documents we had not found by searching companies’ websites. Neither document was easily discoverable by anyone seeking a company’s public policy: both had been prepared by communications departments (a press release and a policy PDF), both were housed in the media section of the website, neither were linked from other parts of their site, and both had only minimal links from the wider internet (on a refined Google search for “pages linking to” the web addresses for these documents). We accepted both, however, as the companies provided evidence that they were publicly accessible at April 2016. Combining the two additional policy documents, and 217 pages of company responses in total, raising over 300 points of contention, we identified seven elements in our database that we changed in the light of critical feedback of our assessments from companies: this was 0.4% (7/1806) of all coded policy elements. The full raw text of all responses is shared as underlying raw data alongside our own data sheets for each company.40
Several companies acknowledged that their policy was ambiguous, or indicated that their policy did not reflect their actions. Specifically: 11 stated that their actions exceeded their public policy commitments, four stated that they will change their public policy in response to our contact, three stated that they planned to change their policy, without being explicit that this was in response to our contact, and four stated that they had already made a change to their public policies on the issues we raised, without being clear whether this was in response to our contact. Some responses about policy changes were problematic. In challenging our coding, some company responses were constructive, making reference to specific phrases in their policies the reading of which they wished to contest. However, most were lengthy, used vague language, and failed to address the specific questions raised. There were recurring themes in responses. Four companies argued that our coding of their public policy was wrong, but the evidence they gave for this assertion was a private document not available in the public domain, such as an internal standard operating procedure. Five companies stated that our coding of their public policy was wrong, but gave no evidence as to why, and their response was not consistent with their public policy. Three companies argued that it was unfair to describe the limitations on their IPD or CSR sharing commitments because they also considered applications for data and documents that fall outside these commitments (this commitment was coded separately in our audit). Notably, Novo Nordisk stated in its response to our coding: “Just for reference, we do not say it anywhere, because we do not want to encourage submission of research proposals that go beyond the stated scope, but we do actually consider all received proposals.”
Discrepancies between industry commitments and individual companies’ commitments
We examined whether all members of the industry bodies made commitments that match the minimum collective commitments made in the “Joint Position” commitments (which cover only trial registration and results sharing) from pharmaceutical industry bodies. Twenty two companies were members of Pharmaceutical Research and Manufacturers of America (PhRMA). Of these, two companies’ policies did not reflect the commitment on registration in the European Federation of Pharmaceutical Industries and Associations (EFPIA)/PhRMA/ International Federation of Pharmaceutical Manufacturers and Associations (IFPMA)/Japan Pharmaceutical Manufacturers Association (JPMA) “Joint Principles” 2005 (and the updated version of 2009); and three did not reflect the same documents’ commitments to share summary results. We asked EFPIA and PhRMA whether the joint position commitments are to be read as binding on all members. They replied: “Although, the Principles are voluntary, the PhRMA and EFPIA Boards encourage member companies to adhere to the principles and confirm the implementation of data sharing procedures through publicly available letters of certification.”
During coding we also found that in addition to membership five companies explicitly stated that they commit to the industry bodies’ “Joint Position” documents (either 2009, 2005, or without specifying which). We therefore additionally tested for discrepancies between these companies’ own explicit commitments and the industry body commitments, to examine whether our coding should be changed or upgraded in light of their stated adherence to an external policy. No changes were warranted. The “Principles on Data Sharing 2013” document from EFPIA and PhRMA commits a company to having an application process for sharing CSRs and IPD, both from 2014 onwards. Again, during coding we found companies who stated that they implemented these principles; we therefore tested to check whether our coding of their policy matched the minimum standard of that external document. Two (Orion and Abbott) were coded as “no” for sharing CSRs on request, based on their own policy documents; arguably these companies could inherit a “yes” for this policy issue, from endorsing the wider industry body document.
Discussion
Company policies and actions on transparency vary widely across all domains. With one exception (journal submission for all trials within 12 months) all aspects of the reference standard were met by at least one company, showing that these commitments are realistic targets, judged as reasonable by at least one company. Company policies commonly used unclear language and exhibited internal contradictions. Many companies lacked commitments on basic issues such as trial registration and the sharing of summary results. Transparency commitments generally did not include trials for off-label uses, even though these are common in clinical practice; and commonly failed to include phase IV trials. Twenty three companies replied when given our appraisal of their policy. Feedback was lengthy, but often ambiguous, and only had a minor impact on our coding: 0.4% of score elements were changed. However, 10 of the companies who responded informed us they were changing their public policies, and four explicitly stated that this was as a result of feedback from our research.
Strengths and limitations of this study
This is the first audit of company policies on trials transparency, an important public health issue. It focused on one methodological and policy issue, measuring clear objective outcomes. All extracted data are shared openly so that others can critically review all coding. Some companies declined to reply when contacted about our assessment of their commitments: however, since this was an audit of public commitments, and company replies identified almost no required changes, we do not believe this substantially affected the accuracy of extracted data. All raw data and source documents are shared in full to allow others to validate or critically review data extraction. As with all projects of this kind, alternative data frames could have been constructed. Ideally, data on various additional features would have been extracted, including: a measure for the time burden on applicants for access to clinical study reports (CSRs) (as some companies share proactively or on simple request, whereas some require full protocol from the applicant, and a curriculum vitae for team members); separate ascertainment of different transparency commitments for trials conducted in different territories, or other classes of trial (eg, investigator initiated trials funded by the company), and more detailed differentiation of limitations on access to CSRs and individual patient data (IPD). However, since it was established during coding that policies were often unclear on basic issues, such additional efforts may prove challenging until companies can move towards less ambiguous policies.
The last limitations relate to scope. Our audit was focused on companies: policies from non-industry sponsors will be addressed in a separate study. In addition, our audit intentionally assessed companies’ commitments, not their performance: delivering the latter would require ongoing public audit of all trials conducted by all companies to establish what proportion had their methods, results, CSRs, and IPD made available. Although desirable,8 such an audit across all aspects of sharing across all trials would require extensive resources. Previous attempts to compare companies’ performance have therefore only focused on registration and the availability of results; additionally they have used either small samples of trials13 or automated approaches processing metadata on larger samples,41 resulting in pragmatic compromises between coverage, accuracy, and precision.
Context of other research
To our knowledge this is the first structured audit and benchmarking project to examine the transparency policies of pharmaceutical companies. The Access to Medicines Index is similar in its intent to grade performance of companies on a complex public health policy issue and has been successful at drawing greater attention to access to medicines and guiding policy discussions on how to facilitate change. Additionally, a 2014 editorial summarises 12 companies’ IPD sharing policies in free text,42 and one team in 200843 reviewed research policies from selected charities, government bodies, and research councils, but not from pharmaceutical companies: it found that 26% mentioned trial registration explicitly, 40% mentioned registration implicitly through reference to other guidelines, and 67% mentioned trial publication explicitly. These figures are consistent with our findings.
Interpretation and policy implications
We found extensive heterogeneity in policies, and several concerning omissions. While larger companies had more complete policies, only 71% of all companies had a commitment to register all trials, the most basic level of transparency; and only 71% had a commitment to share summary results. Company transparency policies overwhelmingly failed to include trials on unlicensed uses of currently marketed products, even though such use is common in clinical practice, and widely promoted outside the law by companies as shown by extensive recent court cases and fines.44 45 46 47 Only 52% included phase IV trials under their transparency policies. Median start dates for transparency commitments were so recent as to exclude the majority of trials on currently used treatments, not just for CSR and IPD sharing (2012) but also for registration and summary results (2005).
We also identified several recurring shortcomings in policies that have not been previously systematically identified. For example, several companies made access to CSRs contingent on approval of a medicine in both the EU and the USA: this means that CSRs would be inaccessible for any indication or medicine approved only in the EU or USA, such as testosterone injections for female sexual dysfunction, or the antidepressant reboxetine; or for treatments marketed in Africa but unapproved in the EU and USA. This is particularly problematic, since treatments rejected by some regulators are likely to be those where the risks and benefits are more closely balanced, and therefore where access to the methods and results of studies underlying regulatory decisions is more important.
The ambiguity in company policies made interpretation and data extraction difficult. Policies were often long, spread over multiple documents or web pages, internally contradictory, and used vague language. This is surprising, given that companies have access to extensive legal and regulatory expertise. The feedback from companies suggests that their own policies often do not reflect their internal practices. This makes appraising their compliance with their own policies extremely challenging both for external actors and, presumably, for themselves. We suggest that company policies should be simple, easily interpretable, and, ideally, standardised. We propose a simple boilerplate transparency policy in box 2.
Box 2: Model pharmaceutical company clinical trial transparency policy
-
Registration We commit to register all clinical trials conducted in all territories, before trial commencement. For clarity this policy includes: phase I/II/III/IV trials; trials in the following territories (specify). This policy has been in place since [date].
-
Summary methods and results We commit to make the methods and summary results of all clinical trials publicly available within XX months of completion. These will be posted on clinicaltrials.gov/our own website. These will be posted as free text/CSR synopses/structured summary data, with accompanying protocols/statistical analysis plans [delete as applicable]. For clarity, this policy includes [delete as applicable] trials on unapproved treatments, trials on unapproved uses of approved treatments, phase I/II/III/IV trials, all trials in all territories/all trials in the following countries (specify). This policy applies retrospectively and covers trials that: started after [date]/completed after [date]/treatments approved after [date]. In addition, all/some (specify which) trials will be submitted to an academic journal within XX months of completion.
-
Clinical study reports (CSRs) We commit to share full CSRs, for all clinical trials with an associated CSR, within XX months of trial completion. These will be posted on: our own website/EU clinical trials registry (but note this only covers EU trials)/clinical study data request site/an independent repository (specify which)]. For clarity, this policy includes [delete as applicable]: trials on unapproved treatments; trials on unapproved uses of approved treatments; phase I/II/III/IV trials; all trials in all territories / all trials in the following countries (specify). This policy applies retrospectively and covers trials that started after [date]/completed after [date]/treatments approved after [date]. Our redactions policy for full CSRs can be read in one document, here [link]. We make CSRs available by posting proactively on a public website [link to website]/a “light touch” requests process open to all, where applicants only specify the document they are requesting [link to requests portal]/a review process where applicants submit CVs and describe why they want access to a CSR [link to request portal and guidance on access control].
-
Individual patient data (IPD) We commit to share IPD on request, for all clinical trials, within XX months of trial completion. Our IPD can be accessed through: our own website/EU clinical trials registry (but note this only covers EU trials)/clinical study data request site/an independent repository [specify which]. For clarity, this policy includes [delete as applicable]: trials on unapproved treatments, trials on unapproved uses of approved treatments, phase I/II/III/IV trials, all trials in all territories/all trials in the following countries (specify). This policy applies retrospectively and covers trials that: started after [date]/completed after [date]/treatments approved after [date]. Our applications process for IPD can be read in one document, here [link].
We intend that the benchmarking exercise reported here can be used by companies to review their own practice, and identify where their policies can be improved: this is supported by the number of companies that responded positively when presented with a structured interpretation of their public policies. We also intend that our findings can be used by regulators, patient groups, professional bodies, ethical investors, and healthcare workers to advocate for improvement at companies with less stringent transparency policies, or to guide procurement decisions.
All results will be presented on a standalone website,40 with accompanying contextual information and links, to increase accessibility for the data and augment impact on practice. We also anticipate that this resource may assist those seeking access to individual trial results, by widening easier access to a summary of the specific commitments made by each individual company. We are already aware of individual cases where company spokespeople have refused access to trial results, and mischaracterised their own company’s policy, in correspondence with systematic reviewers seeking access to information on a specific trial. This may reflect the lack of clarity in some companies’ policies, leading to misunderstandings for staff; or a broader lack of informed public discourse around companies’ commitments. Our publication of company policy commitments in an accessible format may lead to further examples of companies apparently breaching the commitments summarised here; this will provide important information on how companies’ policy commitments are applied in practice.
Future research
Following best practice in assessing impact from audit and feedback we intend to repeat this audit. We welcome methodological criticisms and suggestions to improve the data schema. In addition we are developing a new benchmarking framework to grade actions and policies on transparency by academic journals, academic institutions, and non-commercial trial sponsors.
Conclusion
Public policies on transparency of trials are highly variable between pharmaceutical companies, often ambiguous, and inconsistent with the companies’ description of their own commitments in correspondence. This makes appraising companies’ compliance with their own policies extremely challenging. Company transparency policies should be explicit, with unambiguous statements describing how compliance is monitored. Companies should also aspire to meet the standard of all trials registered, with methods and results made available, and with clear arrangements for sharing CSRs and IPD.
What is already known on this topic
-
The results of clinical trials are commonly left unpublished: this prevents doctors, researchers, and patients from making informed choices about which treatment is best
-
No previous attempts have been made to systematically compare pharmaceutical companies’ policies on access to their trials’ methods, results, regulatory documents, and data
What this study adds
-
Transparency commitments are highly variable between companies, with some making minimal commitments, or none at all
-
Many companies’ policies were poorly worded and internally inconsistent
-
Every element of best practice around transparency was committed to by at least one company; this strongly suggests that they are all deliverable, and that there are no practical barriers to all companies committing to meet all elements of best practice
Footnotes
-
We thank Tracey Brown, Juliet Stevens, Seb Bacon, Anna Powell-Smith, and Till Bruckner for discussions and support.
-
Contributors: BG conceived and designed the study. BG and SL analysed the data. IB and SL collected the policy documents. BG, SL, CH, KRM, and IO extracted the data. BG drafted the manuscript. LS supervised the project. All authors contributed to and approved the final manuscript. BG is the guarantor.
-
Funding: BG is funded by the Laura and John Arnold Foundation to conduct work on research integrity, including this audit. The funder had no involvement in the study design or the decision to submit.
-
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare the following. BG has received research funding from the Laura and John Arnold Foundation, the Wellcome Trust, the NHS National Institute for Health Research (NIHR), the Health Foundation, and WHO; he also receives personal income from speaking and writing for lay audiences on the misuse of science. KRM has received funding from the NHS NIHR and the Royal College of General Practitioners for independent research projects. CH has received grant funding from WHO, the NIHR, and the NIHR School of Primary Care. He is also an advisor to the WHO International Clinical Trials Registry Platform. LS has received research funding and fees from GlaxoSmithKline and AstraZeneca, is a trustee of the British Heart Foundation (BHF), and received research funding from Wellcome, NIHR, the Medical Research Council, BHF, European Union, and a range of charity funders. He is supported by a senior clinical fellowship from Wellcome. SL and IB are employed by Sense about Science, a charity that campaigns on better use of science and evidence; SL coordinates the AllTrials campaign on trial transparency. The views expressed are those of the authors and not necessarily those of any of the funders or institutions mentioned.
-
Ethical approval: Not required.
-
Data sharing: Full coding schema, data sheet, and document archive are available at policyaudit.alltrials.net.
-
Transparency: The manuscripts guarantor affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.