- 1James Lind Initiative, Oxford OX2 7LG, UK
- 2Centre for Research in Evidence-Based Practice, Faculty of Health Sciences, Bond University, Gold Coast, QLD, Australia
- 3BMJ, London, UK
Biased under-reporting of research has been documented for well over two decades and the evidence for it is now overwhelming.1 2 3 4 Under-reporting is research misconduct and has serious consequences.5 6 It leads to overestimates of the benefits of treatments and underestimates of their harmful effects.7 Because of this it puts patients at risk and wastes healthcare resources.
Much of the criticism has focused on commercially funded trials, and justifiably so. There is clear and consistent evidence of under-reporting and manipulation of the scientific literature by the drug and devices industries,4 and industry sponsors most of the world’s clinical trials. But under-reporting is not confined to commercially sponsored trials. Indeed, early examples of failure to publish negative results came from academia.5 8
Nor has academia been any better than industry at cleaning up its act in the intervening decades. Because of trial registration, we can now estimate the magnitude and describe some of the characteristics of under-reporting of clinical trials. Only around half of all registered trials have published at least some of their results, and this level of under-reporting affects most types of trial: early and late phase, large and small, national and international, commercial and non-commercial (figure⇓).9
This matters because participants in clinical trials assume that they are contributing to the advancement of medical knowledge; non-publication of study results negates this reasonable assumption and betrays those who have volunteered.
Non-publication also matters because failure to publish all the results from clinical trials distorts the evidence base for clinical decisions. In a Personal View published in the BMJ eight years ago, the clinical epidemiologist Alessandro Liberati protested that the unpublished results of clinical trials could have informed his choices as a patient with multiple myeloma. “Why was I forced to make my decision knowing that information was somewhere but not available? Was the delay because the results were less exciting than expected? Or because in the evolving field of myeloma research there are now new exciting theories (or drugs) to look at? How far can we tolerate the butterfly behaviour of researchers, moving on to the next flower well before the previous one has been fully exploited?”10 Just over a year ago, Liberati died from the complications of his disease, still waiting for researchers to publish information relevant to his treatment choices.
Many academic trials have failed to report their findings, including important trials supported by major funders. For example, a large trial of adenoidectomy funded by the UK’s Medical Research Council remained unpublished for more than a decade after it was concluded. The study has now at last been reported.11 And this week the BMJ reports on the failure of US academics to publish protocol defined follow-up data from a trial of sentinel node biopsy in malignant melanoma.12
What can explain this failure to publish academic trials? Journals have been blamed for a bias towards accepting positive results, and some of the blame does lie with them. But the evidence indicates that the principal culprits are authors and research sponsors for not submitting reports for publication.13 Financial conflict of interest is well understood as a motive for suppression of unfavourable results from commercially sponsored trials. But what are the motives of authors and sponsors of non-commercial trials? Authors admit failure to write up and submit their results,14 and anecdotes suggest a range of reasons, such as losing interest or moving on to new institutions and projects, poor organisation, inadequate resources, writer’s block, or unwillingness to accept the results of a trial owing to intellectual or reputational investment in the outcome. Despite the billions of pounds wasted, there has been too little systematic effort to monitor the extent of non-publication, let alone investigate the reasons for it.
The responsibilities of authors are clear: the Helsinki Declaration leaves no room for ambiguity. It states that, “Authors have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports . . . Negative and inconclusive as well as positive results should be published or otherwise made publicly available.”15
But authors’ behaviour is unlikely to change without firm action from those who give ethics approval, institutional hosting, and funding support for trials. Research ethics committees were challenged long ago to behave ethically by ensuring that results of trials were published,16 yet these committees have been noticeable by their absence among those exposing under-reporting of clinical trials and taking steps to tackle the problem. It is clear from the figure that academic institutions and funders of research have similarly failed in their responsibilities. There are exceptions, however: the figure also shows that 98% of the studies funded by the National Institute for Health Research Health Technology Assessment Programme have led to the publication of full reports (Ruairidh Milne, personal communication). The programme has achieved this by holding back a proportion of the research grant until a report has been submitted for publication, by chasing authors on a regular basis, and by providing a publication vehicle—Health Technology Assessment—for all trials.
This shows what can and should be done. Information made public through trial registration means that research funders and institutions that continue to under-report clinical trials can now be identified. Patients who are invited to participate in trials should consider the track record of the institutions and funders concerned and refuse to participate unless they receive written assurance that the full study results will be made publicly available and freely accessible (box).
Advice to patients invited to participate in a clinical trial17
Agree to participate in a clinical trial only if: (1) the study protocol has been registered and made publicly available; (2) the protocol refers to systematic reviews of existing evidence showing that the trial is justified; and (3) you receive a written assurance that the full study results will be published and sent to all participants who indicate that they wish to receive them.
A campaign to ensure that all trials are registered and their results published, or otherwise made publicly available, is launched this week (alltrials.net). We invite all BMJ readers to sign the campaign’s petition.
Cite this as: BMJ 2013;346:f105
Competing interests: We have read and understood the BMJ Group policy on declaration of interests and we have no relevant interests to declare.
Provenance and peer review: Commissioned; not externally peer reviewed.