First reviews are published that are based on the “totality of the evidence”

BMJ 2013; 346 doi: (Published 17 June 2013) Cite this as: BMJ 2013;346:f3909
  1. Zosia Kmietowicz
  1. 1BMJ

Scientists and medical publishers have heralded “a historic moment for open science” after a medical device company gave researchers unprecedented access to all the data they had on one of their products for independent evaluation.

The reviews were done by two independent teams using the same data. They both found that recombinant human bone morphogenetic protein 2 (rhBMP-2), an orthobiological agent used in some surgeries to promote bone growth, shows little benefit in spinal fusion compared with iliac crest bone graft or other bone grafts, and that it may cause more harms.

The findings are published in the Annals of Internal Medicine and are based on patient level data from clinical trials conducted by Medtronic, which were shared with researchers through the Yale University open data access (YODA) project.1 2

The data are now being made available to others who want to examine them to answer further scientific questions.

In an editorial, the YODA team praised the work that led to the first ever reviews being published that are based on the “totality of the evidence” and created a new standard in medicine.3

They added: “Let us resolve to have this effort be the first step in the next era of cooperation among industry, academia, clinicians, and the public—one that rewards data sharing, promotes open science, and ultimately makes it untenable to obscure data relevant to the risks and benefits of approved medical products.”

Medtronic was hit by scandal two years ago when the Spine Journal looked at 13 published, industry sponsored trials of rhBMP-2 and reports submitted to the US Food and Drug Administration.4 It found that the trials failed to mention important side effects. The journal’s own review showed that 10-50% of patients treated with the protein had adverse effects, and that surgeons who conducted the trials received payments amounting to as much as $26m (£16m; €18m) from the company and failed to declare them.

Members of the Senate Committee on Finance said that they were “extremely troubled” by the reports and asked for Medtronic to explain itself.4 Two months later, the company announced that it would give Yale University $2.5m to review all published and unpublished data on the safety and effectiveness of its product, the Infuse Bone Graft.5

Yale appointed two teams, the Centre for Reviews and Dissemination at the University of York and the Evidence-based Practice Center at Oregon Health and Science University, to separately assess and reanalyse the patient level data provided by Medtronic.

One review concluded that spinal fusion was marginally improved by rhBMP-2 compared with iliac crest bone grafting (overall fusion relative risk 1.12 (95% confidence interval 1.02 to 1.23)),1 but the other review did not (anterior fusion 1.05 (0.88 to 1.24); posterior fusion 1.16 (0.96 to 1.41)).2 Neither review found that rhBMP-2 improved pain or function.

The York team found that cancer was more common after rhBMP-2 use (relative risk 1.98 (95% confidence interval 0.86 to 4.54)). However, the small number of cases (20 of 694 patients receiving rhBMP-2 v eight of 608 receiving iliac crest bone grafting) meant that this result was inconclusive. Similarly, the Oregon team found an increased risk of cancer at 24 months, although the overall cancer risk was low.

The Oregon team said that lack of blinding of surgeons, patients, and outcome assessors were the main sources of bias in the Medtronic studies.

Lesley Stewart, who led the York team, said: “This has been a ground breaking project. As well as addressing an important clinical question, the project has set a new standard in transparency and release of trial data for independent scrutiny—a development that is in the public interest and an issue that is currently being examined by the UK Parliamentary Select Committee on clinical trials.”

Editor-in-chief of Annals of Internal Medicine, Christine Laine, wrote that the fact that two independent groups arrived at essentially the same conclusions should greatly temper enthusiasm for rhBMP-2.6 This first YODA project “is a novel exercise that illustrates the value of evidence synthesis, data sharing, peer review and editing, and reproducible research in helping us get closer to the truth,” she added.

Richard Kuntz, from Medtronic, said that the company agreed to the project because of its commitment to explore objective evaluation processes of its products. Although open analyses may not reinforce the company’s own evaluations and conclusions, “all stakeholders should benefit from a transparent process that produces reliable information about the benefits and harms of the products being studied,” he wrote.7


Cite this as: BMJ 2013;346:f3909


View Abstract

Log in

Log in through your institution


* For online subscription