Intended for healthcare professionals

Observations Reality Check

It’s time to rebuild the evidence base

BMJ 2011; 342 doi: https://doi.org/10.1136/bmj.d3004 (Published 25 May 2011) Cite this as: BMJ 2011;342:d3004

It's time to rise the wider question

The influence of industry or government sponsors on the design of the
trials and on selective publication of the results is now well
established. It is why Cochrane consumers for a long time asked for the
disclosure of the funding of the original studies included in the Cochrane
reviews and potential conflicts of interests of authors of original
research. Once the Cochrane review was based totally on the producer
funded trials, half of them not published, had persisted in the Cochrane
library for years. Later it was shown that this specific drug is not as
effective as it was known from the producer initial trials summarized in
the review. Albeit it is a rare example, it is an example of how industry
sponsored trials biased the systematic review of the drug efficacy.

But is the source of funding the only visible source of biased reports? Of
course not. There is another dimension for the skeptic view - the quality
of study and the honesty of the report.

The quality of the study is mostly visible through the quality of the
report, and the report may be beautified by the author. As a result, the
best efforts to evaluate the likelihood of bias in the study are at best
limited.

The study report may be dishonest on the large scale - the report of the
totally fabricated study or the total plagiarism of the previous study. In
this sense the repeated publication of the study distorts the world
research record in the same way as publication of the plagiarized report.

The diligent systematic reviewer, who includes in the review all relevant
studies, but does not pay attention to the potentially fabricated or
manipulated studies or to the source of funding or conflicts of interests
of authors is providing a suboptimal service to society. As I well know, the major barrier for Cochrane reviewers to flag in their review
the studies tainted by conflict of interest or studies presenting the
possibly fabricated/manipulated data is the concern that such evaluation
of the primary studies may lead to unpleasant contacts and probably to
litigation.
If systematic reviewers will not report the found cases of plagiarism, data
manipulation and fabrication - who can do it? Nobody. Authors of
systematic reviews are in unique position to detect research misconduct
and initiate the actions necessary for cleaning the research record. What
they need - the urgent development of the guidelines on how to report any
concern in the quality of the primary data. The quality in the wider
context, not only study funding.

Competing interests: No competing interests

12 June 2011
Vasiliy V. Vlassov
President
Society for Evidence Based Medicine, Russia