Labeling Studies with a Reliability Factor?
Unreliable knowledge production pays off, as we can conclude from the
billions of dollars spent oseltamivir (Tamiflu), without convincing
evidence. Studies on oseltamivir are considered less reliable due to
commercial sources of research, a lack of transparency of data and the
absence of external scrutiny and review (doi:10.1136/bmj.b5405).
Therefore, a plea for more reliable data on oseltamivir, and more
transparency on raw data seems very straightforward. However, the ethical
question arises: Can we still legitimize commercially funded research?
Patients given treatment based on less reliable studies are put at risks.
And what to think of the balance between risks, burdens and benefits for
trial participants, when possible benefits are questionable? Increased
scrutiny and external control would be logical policies to suggest.
However, most Western countries already have a very mature regulation
system in place, and obviously still essential information does not meet
the eye. Are we then creating sham-safeties by such regulative policies?
Suggested public availability of raw data might increase possibilities for
external control. However, the problem of being dependent for information
on industry cannot be solved this way, and checking data analysis would
still require a lot of time and effort for researchers and/ or journalist
to investigate. In the meantime we might consider the value and
possibility of marking articles with a ‘reliability’ factor based on the
level of credibility attributed by a journal to a specific article. Such a
reliability factor could be based on criteria like transparency of data,
accessibility for external controls, financial and non-financial
interests, and perhaps also more subjective criteria like reputation of
the funders, authors and researchers. Adding such a factor would certainly
not solve reliability problems, but could make differentiations in
reliability of articles (and journals) more transparent for researchers,
physicians, patients and IRB or MREC members etc. And moreover, this would
hopefully stimulate industry to actually show transparency and more
general publicly responsible conduct. EQUATOR, a network concerned with
accurate and transparent reporting of health research, could play an
important role in the development of criteria for different reliability
Website www.equator-network.org; visited 14-12-2009