Cochrane proposes further limits on commercial funding
BMJ 2004; 328 doi: https://doi.org/10.1136/bmj.328.7436.366 (Published 12 February 2004) Cite this as: BMJ 2004;328:366
All rapid responses
We write from the editorial base of the Cochrane Pain, Palliative
and Supportive Care Collaborative Review Group (PaPaS CRG)
regarding the debate that has been stirred by the above-titled
news item from Jeanne Lenzer. We wish to address the criticisms
that have been made of our editorial processes in relation to the
publication of Cochrane reviews on eletriptan and rizatriptan for
acute migraine.
Firstly, we would like to emphasise that the two reviews in question
are not the only work from our group that have assessed
interventions for headache and migraine. The collaborative review
group currently has 25 Cochrane reviews investigating headache
and/or migraine either in progress or published in The Cochrane
Library. (All registered Cochrane review titles can be viewed at
http://www.cochrane.no/titles/). These include reviews on two
other triptan drugs (sumatriptan and naratriptan), along with
reviews of older drugs and complementary and alternative
therapies.
Secondly, we agree with Jeanne Lenzer that there is a need for 'a
mechanism by which health priorities – rather than proprietary
research priorities – are systematically researched and reported’.
The editorial board of PaPaS has made efforts at precisely such
priority-setting for headache reviews, beginning in 2000, and has
revisited and updated its list of priority topics annually ever since.
We took a variety of issues into account in constructing our original
priority list, including, for example, the WHO essential drugs list.
Triptans were and remain a priority because, as a class, there is
good and consistent evidence for effectiveness in the treatment of
acute migraine and compare favourably with alternatives in head-
to-head trials. However we have no resources to fund reviews
according to any priorities we might set.
Although we are guided in our editorial decisions in part by our
priority list, when it comes to the actual recruitment of review teams
to prepare Cochrane reviews, PaPaS does not ‘commission’
reviews from particular teams but registers titles as and when
interested researchers contact us, providing the title has not been
previously assigned. We are also guided by the following
Cochrane Collaboration policy (free to view in the Cochrane
Manual at www.cochrane.org):
4.3 [Cochrane ] Collaborative Review Groups should avoid
having a small number of reviewers, each of whom is responsible
for a large number of reviews. This does not promote diversity of
opinion in producing reviews and also may cause problems when
it comes to a single reviewer keeping a large number of reviews
up to date. Collaborative Review Groups need to set their own
limits, but five reviews per reviewer might be a reasonable
maximum limit. It is also preferable to have more than one
reviewer working on each review.
In accordance with this policy, when considering the approach
made by Andrew Moore’s research team to prepare Cochrane
reviews on drugs for migraine, our editorial board had to consider
the number of individual reviews this might entail. The decision by
the editorial team to allow Dr Moore to register only two titles was
made on the basis of the number of Cochrane reviews his team
already had in progress. The decision to focus on triptans was
influenced by the priorities set by the PaPaS editorial board, as
described above. The choice of eletriptan and rizatriptan, among
the various triptans offered, was influenced by two factors. First,
expressions of interest in preparing reviews of sumatriptan and
naratriptan had already been received from other research groups.
Second, the proposed eletriptan and rizatriptan reviews were
comprised entirely (eletriptan) or partly (rizatriptan) of data that had
not (at the time) been published, but which had been provided to
Dr Moore’s research team by the manufacturer (with advance
agreements regarding the completeness of disclosure of the data
and protections for the independence of the authors'
interpretation).
We did not consider withholding the eletriptan and rizatriptan
reviews until reviews of other triptans could be completed. Our
practice within PaPaS is to support each review team through the
Cochrane process until either a review is published or the team
withdraws. For various reasons, it is inevitable that some reviews
reach completion faster than others, and we do not have a policy
of holding completed reviews until others can be finished. That
would be illogical and unfair to authors, and the reviews would be
out of date by the time they were eventually published.
We believe that the job of the editorial review team is to help
authors prepare reviews and ensure that reviews are done with
appropriate methodology and include all relevant studies. It is
also the editors' responsibility to protect against bias, in so far as
this is possible. Toward these ends, we reviewed agreements
between the authors of the eletriptan and rizatriptan reviews and
the drug company sponsoring their original review; consulted with
the UK Cochrane Centre about the propriety of accepting the
topics; and, given the funding issues, wrote a carefully worded
conflict-of-interest statement for inclusion in the reviews.
Additionally, we arranged for independent peer review of the
authors' findings; carefully reviewed their methodology and results
ourselves; and compared the completeness, accuracy, and
conclusions of the reviews with other independent reviews of
triptans.
Having thus followed all established procedures within the
Collaboration, and having assured ourselves of the quality and
independence of these high-priority reviews, we elected to accept
and publish them promptly.
We have attempted to be transparent in dealing with all the
significant issues we perceived in connection with the publication
of these reviews. PaPaS received no monies from industry in
relation to these reviews, and the authors of the reviews were not
funded to convert their original work into Cochrane format. The
Co-ordinating Editor of PaPaS discussed the issue of the financial
sponsorship of these two reviews at some length with Hilda
Bastian at the time of first publication. Since that time, there has
been no request to us for information or clarification on this issue,
and no critical comment has been received via the Cochrane
Library’s ‘comments and criticisms’ facility.
More definitive guidance from the Collaboration’s Steering Group
regarding how to navigate the complex relationships between
drug manufacturers, reviewers, and editors in the generation,
evaluation and publication of Cochrane reviews is awaited.
Signed
Philip J Wiffen MSc MRPharmS
Co-ordinating Editor. Cochrane Pain, Palliative and Supportive
care Collaborative Review Group. Oxford UK
Douglas C. McCrory, MD, MHS
Lead Editor for Headache Reviews
Center for Clinical Health Policy Research, Duke University
Medical Center,
and the Center for Health Services Research in Primary Care,
Durham Veterans Affairs Medical Center, Durham, NC . USA
Competing interests:
Editors of the Cochrane
Pain, Palliative and
Supportive Care Group
Competing interests: No competing interests
I find it a bit disturbing that I, as a journalist reporting on the
controversy about industry funding of the Cochrane Collaboration, should
be attacked by Dr. Andrew Moore and Professor Henry McQuay for supposedly
writing an “inaccurate” report. It is far easier, it appears, for Moore
and McQuay to attack a journalist than it is to counter the statements
made by the authorities I quoted (and quoted completely accurately, I
might add). Do Dr. Moore and Prof McQuay want to claim otherwise? I hope
this merely represents some misunderstanding on their part, rather than a
diversionary tactic to deflect attention from important issues.
Moore and McQuay dispute two fundamental ideas made by the
authorities I quoted. First, they claim that I accused them of a “big
invisible bias.” It was of course Hilda Bastian, the former consumer
advocate for the Cochrane Collaboration, rather than me, who pointed out
that selective reporting by the Cochrane on only two triptans, among a
field of triptans, created the possibility of a “big invisible bias.” Her
concern was echoed by many people I interviewed, and although I was not
the author of this sentiment, I must admit that it seems to me to be an
extremely reasonable concern – especially as one of the two drugs
reviewed, fortuitously enough, just happens to be sold by one of the
industry sponsors of the review.
Moore and McQuay, two of the four authors of the Cochrane review in
question, acknowledge that they receive funding from drug companies, but
offer as evidence of their own lack of bias an article they wrote in Pain,
which reviewed all of the triptans. I won’t address here whether or not
receiving drug company funding creates a conflict of interest for the
authors, because it seems to me the “invisible bias” of which Ms Bastian
warns has to do with the Cochrane itself, and not the authors of this
review. In that regard it is worth noting Moore and McQuay’s comment that
they offered the “whole work” to the headache section of the Cochrane
Collaboration, “and they [Cochrane] chose to take just two drugs,
eletriptan and riaztriptan.”
This is where the “big invisible bias” appears. What might explain
the Cochrane’s rationale for selecting only two triptans to review?
Herein lies the rub (and perhaps the reason for the diversionary attack on
the conveyor of this saga). It is here, it seems to me, that Hilda
Bastian’s concerns resonate so loudly. Why were only two triptans
selected? Were all the other triptans less effective? More dangerous?
More expensive? So much so that it wasn’t worth even reviewing the
evidence for physicians and the public to see for themselves? And could
it have anything to do with the fact that the review itself was
substantially funded by one of the drug’s manufacturers?
I personally don’t know the answer to these questions, but I do
believe that it is critical that they be asked. To claim that funding has
no effect on the selection of drugs reviewed (when some are reviewed and
others aren’t) proves nothing. Unless one truly believes, as Moore and
McQuay profess to believe, that there is “no good empirical evidence that
the results of reviews sponsored by industry differ from results of the
same reviews sponsored by not-for-profit sources.”
Really?
If Moore and McQuay, so outspoken on this subject, are totally
unaware of the “avalanche” of studies (in the words of Dr Drummond Rennie)
showing how strongly commercial funding affects published conclusions,
then I refer them to organizations such as Healthy Skepticism, Health
Action International, and Nofreelunch, or to the special article
collections in any number of medical journals such as the BMJ, JAMA, CMAJ,
etc., on bias, in which they which kindly categorize the astonishing
number of ways in which “gifts” from industry influence scientific
pronouncements.
The failure to report certain studies or to give only partial reviews
of data because they don’t promise a good “return on investment” is well-
described in a recent editorial1 – but one of the most remarkable aspects
of bias seems to be that those affected remain blissfully blind to their
own biases: (other doctors, most everyone acknowledges, might be affected
by financial conflicts of interest – “but I’m certainly not”).2
Drug company funding of research is increasing geometrically. Such
funding takes many forms, including funding of third party intermediaries
such as universities, contract research organizations, and even now, one
of the most prestigious, and purportedly “disinterested” organizations,
the Cochrane Collaboration. This may be why so many people with whom I
spoke expressed a sense of consternation.
But if I am allowed to express a personal opinion here (which I did
not do in my article, despite the implications of the attack by Moore and
McQuay), I would suggest that even if drug companies are no longer allowed
to fund Cochrane, and reviews are merely based on research as currently
done and reported, then drug company influence (the big invisible bias)
will persist mostly unabated, unless the Cochrane, among others, can
establish a mechanism by which health priorities – rather than proprietary
research priorities – are systematically researched and reported.
It is hard to overestimate the powerful, although occasionally
invisible, influence of for-profit companies on not only the research that
is done, but also the way it is interpreted and spun, regardless of how
“disinterested” most observers claim to be. No amount of diversion by
attacking a journalist can hide this fundamental difficulty of for-profit
corporate influence on health care today.
Jeanne Lenzer, medical investigative journalist
Reference List
(1) The "file drawer" phenomenon: suppressing clinical evidence.
Canadian Medical Association Journal 2004; 170(4):437.
(2) Dana J, Loewenstein G. A Social Science Perspective on Gifts to
Physicians From Industry. JAMA 2003; 290(2):252-255.
Competing interests:
I was the author of the article.
Competing interests: No competing interests
As the person responsible for the contested quote about "a big
invisible bias" in Cochrane's offerings of evidence on triptans for
migraine, my response is that the statement was accurate (as was the
reporting of the statement).
I was not speaking at all about potential bias (existent or not)
within the reviews themselves - there is no need to argue this on a case
by case basis any more, as systematic review has shown that readers are
right to be suspicious when they see drug company money attached to
research related to the company's products.
I referred instead to the invisible bias, about what evidence is
brought to our attention as readers who may act on health information.
Anyone who knows how many triptan drugs were on the market in 2001 and
then looked at The Cochrane Library in 2001 could see there was a bias -
for some reason - in what Cochrane was offering. There still is in 2004.
Cochrane's information menu to the public was skewed in favour of one
company's products. This is separate from the issue of quality of the
science.
Overall, we have more medicines-based evidence than we do evidence-
based medicine. For Cochrane, with no other over-arching priority-setting
mechanism than people's enthusiasm, capacity is often generated by funds:
it is a free market forces situation. In the triptan case, in some way,
money created the capacity that ultimately gave pre-eminence to the
products of one company.
These biases were not immediately visible to readers who had two
particular interventions for migraine drawn to their attention on Issue 3
of The Cochrane Library in 2001. Some might have thought these must be
potentially the two most important triptans, or the most effective, or the
most widely used, the most widely available even, or the best value for
money - or perhaps priority was given, they might have thought, because
those two had more safety concerns. But that wasn't why those two drugs
got (and kept) Cochrane limelight.
Yes, the issue of conflict of interest is not simple. But should
readers see drug company investment as a 'caveat emptor' to information
produced? The evidence tells us they should. Is this different to other
conflicts of interest? Yes it is. We all have a variety of interests, and
some of them conflict and can do battle within us, consciously or
subconsciously. Listed for-profit product manufacturers are on their own
in a particular category though: those corporate bodies whose
responsibility is to ensure their shareholders get the best possible
return on investment. No conflict there - just one interest. That's not
like everybody else. And no one else has the capacity to skew the agenda
on quite that scale, either.
Competing interests:
None declared
Competing interests: No competing interests
Andrew Moore states that the Lenzer report was inaccurate.
He states-: "we are accused of contributing a “big invisible bias” by
publishing reviews of only two triptans, with part funding from Pfizer."
Looking again at the Lenzer news report, I could not immediately
identify a direct connection between the fact that the published Cochrane
report on the treatment of acute migraine headaches apparently only
contained a review of two triptans, and Lenzer's statement further in the
article that uses the term invisible bias-: "If you allow free market
forces to set the agenda, you end up with this big invisible bias—and
that’s even bigger than individual bias and individual reviewers."
Now that I can identify a statement in Lenzer's new report referring
to the term "invisible bias", and now discovering from Andrew Moore's
rapid response letter that his group performed a more extensive review of
other migraine drugs, I am trying to understand why the Cochrane Review
apparently only included two triptans in its final review.
Andrew Moore stated in his rapid response letter-: "We agree with
Professor Ian Roberts, quoted in news roundup as saying “Everyone has
interests. The point isn’t to deny those interests but to make them
explicit." The sponsorship in these migraine reviews was explicit. The
decision to take just these two reviews was a Cochrane decision, discussed
before publication with the UK Cochrane Centre. Any potential bias was
very far from being invisible. It is hard to see how you would make it
more visible or more explicit."
If I understand that statement correctly, Andrew Moore is stating
that the decision to take just those two triptan reviews was made by the
Cochrane Center before publication, and that any potential bias was far
from being invisible. From my perspective, the bias is still invisible
because I cannot understand why the Cochrane Center selectively chose the
reviews of those two triptans - eletriptan and rizatriptan.
I also think that it is easy to see how the matter could be made more
visible or more explicit. Someone needs to explain the source of the bias
that made the Cochrane Center only selectively choose part of the larger
work -- "Systematic review of treatments for acute migraine" -- the part,
which involved the two triptans - eletriptan and rizatriptan.
Jeffrey Mann, MD.
Competing interests:
None declared
Competing interests: No competing interests
We read the bmj.com 14 February news roundup piece by Jeanne
Lenzer with some sadness. This inaccurate report has the
potential to further confuse the Cochrane debate about review
sponsorship.
Your correspondent did not check her facts with us, yet we are
accused of contributing a “big invisible bias” by publishing reviews
of only two triptans, with part funding from Pfizer.
The fact is that we reviewed all acute migraine treatments then
available, funded by both for-profit and not-for-profit sources, and
this review was published subsequently in full (AD Oldman, LA
Smith, HJ McQuay, RA Moore. A systematic review of treatments
for acute migraine. Pain 2002 97: 247-257). We offered the whole
work to the headache section of the Cochrane Collaboration, and
they chose to take just two drugs, eletriptan and rizatriptan.
The conflict of interest statement for these Cochrane reviews on
eletriptan and rizatriptan makes it clear that the work on these two
drugs was part of a comprehensive project which would cover all
treatments for acute migraine then available, and for which we had
useable trials (randomised, double blind, International Headache
Society diagnosis, sensible outcomes at sensible times).
"Systematic reviews of rizatriptan and eletriptan were undertaken
as part of a wider piece of work largely, but not entirely, funded by
the manufacturer of eletriptan (Pfizer). The contract specified that
all data on eletriptan (mostly unpublished) would be made
available and that authors had freedom to publish results. The
reviewers made their own choice about trials included or excluded
and outcomes chosen or reported. Pain Research Unit resources
were also used in the preparation of the review." These issues
were discussed with the UK Cochrane Centre before publication.
We agree with Professor Ian Roberts, quoted in news roundup as
saying “Everyone has interests. The point isn’t to deny those
interests but to make them explicit." The sponsorship in these
migraine reviews was explicit. The decision to take just these two
reviews was a Cochrane decision, discussed before publication
with the UK Cochrane Centre. Any potential bias was very far from
being invisible. It is hard to see how you would make it more
visible or more explicit.
The implication of Jeanne Lenzer’s piece is that reviews
sponsored by industry are necessarily biased in favour of the
company’s drug. We have no good empirical evidence that the
results of reviews sponsored by industry differ from results of the
same reviews sponsored by not-for-profit sources. Without industry
we would not have the individual patient data necessary both for
methodological research and for reviews of new drugs.
Independence of review authors from industry sponsors should be
(and was for us) written into the contractual arrangements between
the academic institution and the company.
An informed debate is what is needed, not inaccurate reporting.
Competing interests:
RAM and HJM have been
consultants for
pharmaceutical companies
and other bodies, and have
received research grants
from industry, government,
and charities
Competing interests: No competing interests
That the BMJ pays so much attention to the Cochrane Collaboration's
(CC) internal debate on Conflicts of Interest (CoI) and commercial
sponsorhip is a good sign and confirms the CC's growing influence and
importance.
To bring to meaningful and appropriate conclusions and policy
actions, however, this debate should be given enough time and should not
be overinfluenced by undue expectations, excess emotions being reported
with too much unhealthy sensationalisms.
Whatever the CC would decide its future policies should be, in fact, this
will not the last words nor the solution to an issue that is pervading all
aspects of modern world.
Having had the priviledge, as a CC's member, to express my personal
views on this internal debate I can tell that it has already been a
tremendous opportunity to unheart all the complexities of CoI in a much
richer way than it is reported in your News session.
Those waiting for the conclusions of this debate - which is the
Collaboration's Steering Group duty to cristallize -should therefore know
that:
a) the discussion has already gone well beyond a black/white statement
about sponsorhip from commercial sources;
b) many important issues are being discussed as part of this debate
including the need for better prioritisation and transparency in setting
the research agenda and the need of separating reviews from all possible
sources of CoI.
Having said that, I'd rather wish to express my hope that the CC's
example will be followed by the largest possible number of scientific
organisations which are legitimately expected to play their role in
fighting CoI as a fundamental treath to the credibility of the whole of
medicine and medical care.
Competing interests:
None declared
Competing interests: No competing interests
"Keep your friends close, but keep your enemies closer", said
Brando's character, Corleone, in the Godfather. The real problem, though,
is knowing the difference. Either we are all conflicted (and thus
potential enemies), or we will surely go mad! There are strings attached
to everything, and a financial competing interest is only one of them.
Intellectual bias, preference, faith, stupidity and our own ignorance of
the truth are all sources of conflict and are unlikely to go out of
fashion, no matter how much we try to mask them with a veneer of
'correctness'. It would be much wiser for the Cochrane folk to come to
grips with this, and create a realistic transparent system, rather than an
artifical world of paper saints and pariahs.
Competing interests:
Completely shot full of conflict from normal human passions, all the way to the need to earn a living.
Competing interests: No competing interests
The power of economic drivers
The responses to Jeanne Lenzer's original piece have provided an
interesting explication of the socioeconomic drivers in the production of
Cochrane reviews. It's important to remember the context for this
discussion: what would the impact on the agenda be of opening the process
of independent review of research to more manufacturer-driven influence?
In 2001 when reviewed by Cochrane, eletriptan was not even yet
approved for use in the USA. There were 7 triptans altogether, and
eletriptan's market share was negligible. Its share of published Cochrane
reviews on triptans in 2001 however was 50% - and that was at the time it
was making its push to be allowed onto the US market and it had only
recently come onto the market in Europe. Enough said.
And the other 5 triptans not reviewed in 2001? It was nearly two
years until one of them reached Cochrane publication. And only one more is
even on the agenda (having reached only protocol stage as of early 2004).
So as of early 2004, 3 of 7 triptans have full Cochrane reviews published
and only 1 more is in the pipeline.
That's a bias. It's to do with money, and who could afford to fast-
track Cochrane reviews.
It's important to remember the context, and just what kind of money
we're talking about here. In drug company parlance, triptans are a market
"trending upwards". According to a February press release from the
manufacturers of sumatriptan, it "is the migraine market leader, boasting
sales of $1.19 billion in 2002". Even late-comer eletriptan, only gaining
FDA approval in December 2002, was already posting US$9 million in sales
for the second quarter of 2003.
A Pharmacor study projected that the migraine drug market would
double from $2.86 billion in 2002 to $5.6 billion by 2012, with triptans
holding 75% of that market - and that's only the income from 7 countries.
But enough - trying to grapple with numbers with so many zeros is
giving me a headache!
Competing interests:
None declared
Competing interests: No competing interests