Pharmaceutical industry sponsorship and research outcome and quality: systematic review
BMJ 2003; 326 doi: https://doi.org/10.1136/bmj.326.7400.1167 (Published 29 May 2003) Cite this as: BMJ 2003;326:1167
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Misleading meta-meta-analysis
Those of us working at the coalface of clinical practice struggle to
ensure our work is evidence based and that are critical appraisal skills
are up to the mark. We toil long hours and look forward to our weekly
evidence based journal club where a mixture of grades and levels of
experience come together, usually over a sponsored sandwich, to do battle
with another research paper. However, all too often these articles are
over long, poorly written and insufficiently edited. Our only respite
tends to be when an article from the BMJ is selected – telling us all we
need to know in a punchy 2000 words. When the train spotters amongst us
want more – there is the electronic long version
We enjoyed the timely and illuminating themed issue on the
pharmaceutical industry and the medical profession and had selected Joel
Lexchin et al’s systematic review for critical discussion. We feel badly
let down by the ‘paper short’ version of this review, which was at best
misleading, and at worst confusing and inaccurate. The whole paper reads
as if it were a review of primary studies, rather than what it was – a
review of other peoples reviews. There were a number of methodological
issues, including (1) double counting of evidence and (2) reviewing non-
systematic reviews that cast doubts on the validity of this approach. It
was difficult to tell what the authors had actually done in the shortened
version of the paper. We wonder if this was a ‘cut too far’ and would
urge the BMJ editorial staff to consider whether the readability and
clarity of papers is being sacrificed in search of brevity when producing
shortened paper versions.
Yours sincerely
Charlotte Ford
Senior house officer in psychiatry
Leeds
Simon Gilbody
Senior lecturer in mental health services research
Competing interests:
None declared
Competing interests: No competing interests
Our work about bias in pharmacoeconomic studies (1) is included in
the review by Lexchin et al (2). In the study, we found that 22 (92%) of
the 24 studies published in a specialized journal reported positive
results, 83% of them being sponsored by the pharmaceutical industry; while
only 34 (49%) of the 69 studies published in general medical journals, 74%
of them being financed by government agencies, reported favorable results
for the new intervention tested (1).
Our results suggested that bias in economic analyses could operate in
different directions. Governments and payers have a legitimate interest in
reducing their health expenditure budget, and this might be considered a
source of potential bias to show that the cheapest drug is the most cost-
effective, particularly so when these institutions are financing the
medicines. In such cases, an "unfavorable" result could result in the
withdrawal of finance for the new technology with the subsequent financial
saving. In our work, we stated that, in pharmacoeconomic evaluations it
would be necessary to reconsider the meaning of the term "positive result"
so as to indicate the result which coincides with the sponsor's
expectations (in opposition to the classical definition of the term: the
results which shows a difference in favor of the new therapy).
Surprisingly, the main results of our paper (relative low number of
positive results in pharmacoeconomic articles funded by “independent
sources”) are not discussed at all in the paper by Lexchin et al (although
they are presented in Figure 2), probably because these results are not
consistent with the commonly accepted opinion. We think that this partial
presentation of the results represent a bias. In their article, Lexchin et
al assert that “authors and editors should consider including a statement
concerning prior beliefs of the investigators about the uncertainty of the
treatments that are reported” . We endorse this proposal, and we would
recommend that a similar statement is included in articles that analyze
bias, but in this case, the statement should be on the prior beliefs of
the investigators about the direction of the bias.
This is not the first time that we perceive this kind of bias in
paper about bias. One of the papers also mentioned in the review (3)
stated that pharmaceutical company-sponsored studies were more likely than
non-profit-sponsored studies to overstate quantitative results, i.e., a
favorable qualitative conclusion when quantitative results were neutral or
unfavorable”. Paradoxically, the authors based the affirmation in results
that were not statistically significant.
Probably, the authors of the review will argue that the objective of
the study was to analyze biases of studies founded by pharmaceutical
companies and not about biases of studies funded by “non-profit”
organizations. They are right, but the systematic tendency to analyze the
problem of industry-funded research and not the problems of other kind of
research may be considered also a kind of bias.
We think that measures should be taken to minimize biases in clinical
research. The pharmaceutical industry has already started to work on that.
The recent policies encouraging publication of negative results by PhRMA
(4) are good examples. Hopefully these policies have a double effect: to
decrease publication bias in studies sponsored by the industry and to
start to analyze the problem of bias from a wider perspective.
REFERENCES
1. Sacristán JA, Bolaños E, Hernandez JM, Soto J, Galende I.
Publication bias in health economic studies. Pharmacoeconomics 1997; 11:
289-90.
2. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry
sponsorship and research outcome and quality: systematic review. BMJ 2003;
326: 1167-76.
3. Friedberg M, Saffran B, Stinson TJ, Nelson W, Bennet CL. Evaluation of
conflict of interest in economic analyses of new drugs used in oncology.
JAMA 1999; 282: 1453-7.
4. PhRMA Principles on Conduct of Clinical Trials and Communication of
Clinical Trial Results, 2002
(http://www.phrma.org/publications/quickfacts/20.06.2002.428.cfm)
Competing interests:
Jose A Sacristan and Jesus M Hernandez are employees of Eli Lilly & Co
Competing interests: No competing interests
“Here is no water but only rock … amongst the rock one cannot stop or
think”
T.S.ELIOT
The Waste Land
I welcome the article by Lexchin et al. published in the current
issue of the journal. It is common place that the progress of medical
science has become increasingly dependent on pharmaceutical industry and
this seems to have direct implications for public health affairs. The link
between medicine and pharmaceutical industry has directly affected the way
that medicine is practiced and also the quality of medical care. In an
increasingly globalised world, if medical research and practice is
dictated by the needs of industry, this suggests crisis. I really wonder
if there is still enough space for some independent research in academic
laboratories. On the other hand, it is dangerous when medical research
narrows its concentration into areas directed by the profit motive of
pharmaceutical industry. I think this, finally, underlies the
helplessness of the individual.
The medical community itself knows better than anyone else how deeply
these ties traumatise the integrity of medical profession, and also, if
efforts and codes of practice failed, what the potential consequences
might be. I believe that the central question is not whether someone
should tell clinicians and researchers what to do or not to do, but who
impose such limits and directions and by what criteria. And if there will
not be a clear distinction between medicine and pharmaceutical industry
the results of this entwinement cannot be predicted. It is a matter of
medical culture and, indeed, if doctors learn to shrug their shoulders
about these issues the future of medicine seems dull, and, then, perhaps
the price for patients and society may be high.
Competing interests:
None declared
Competing interests: No competing interests
I agree with the other rapid responses that there are other reasons
inherent in publication bias in such studies, including interest and
motivation of the authors of non-drug company sponsored trials, to seeing
them eventually published somewhere. But surely a more pernicious attitude
is the ability of sponsors to veto publication of any results? Although
pharmaceutical companies have been roundly critised in this week's BMJ,
government sources are probably even more guilty. I know of several
government ministry funded studies, superficially awarded as grants where
the researchers have no right of publication.
Surely the only answers to this are for ethics committees to refuse
permission for such clearly non-independant studies and for the MRC and
NHS RandD to actually fund investigator-driven practical clinical studies?
Competing interests:
None declared
Competing interests: No competing interests
One of the possible explanations offered for the bias in drug-company
funded research is that pharmaceutical companies may selectively fund
trials on drugs that they consider to be superior to the competition. This
is dismissed by saying that researchers cannot predict results of trials
in advance.That is undoubtedly true, but not always true.
When applying for funding from drug companies, there's a fairly
competitive and rigorous process to go through. The pharmaceutical
companies are commercial entities, and as such would be foolish to choose
to fund research where the literature review providing the rationale for
the study to be funded indicates an unfavourable (to the product and
company) result will be the probable outcome. Researchers are as aware of
this as the companies themselves, and will apply for funding to the
company whose product is likely to be shown to best advantage in the
study, because they believe that is their best chance of obtaining the
funding.
Therefore, it may be the researchers, rather than the drug companies,
who are being selective. This should be quite simple to test for - simply
by questioning applicants for funding, or looking at all proposals, the
unsuccessful as well as the succesful. To (mis)quote, 'never ascribe to
malice what can best be explained by cupidity'!
Competing interests:
I am employed in an organisation that critically appraises medical literature
Competing interests: No competing interests
Dear Sir,
The article by Lexchin et al is interesting but does not address two
major issues. Firstly as a metanalysis it is vulnerable, as are all such
publications to publication bias itself. The authors do not report whether
any of the studies they examined were themselves drug - company sponsored.
I assume none of them were so funded. Adopting a cynical view - which is
de rigure in any discussion of bias - there seems a fairly strong
potential bias for publication of studies that discover bias, if there is
no contrary pressure.
Secondly, the discussion does not cover what is in my view a major source
of such 'bias' in original studies. Take the case of a drug that is
about to complete approval. Assuming crudely, that it is either successful
or not, then I would argue that if it is not successful - ie does not
gain approval then there are a number of strong forces stopping
publication that could not be seen by a reasonable person as bias.
1) The drug company will now have no interest in the drug, and would be
extremely reluctant to fund conference travel for example.
2) The researchers would realise that such a publication is likely to be
less valuable to them than a postitive one, and would be reluctant to
"waste time" on a publication that will not lead to follow ups.
3) Any journal would be less likely to publish the results of a study on a
drug that is not going to be available.
However, taking the example where a non- drug company funded study
occurs for a new technique or treatment, the researcher does at least gain
credit for the intellectual effort made in deciding to invent the
treatment, and such a publication will assist the researcher in terms of
prestige etc. This does not occur when the invention was made by someone
else - for example a drug company, and even the protocol was devised by
them.
Undue influence by drug companies may be a problem, but I am
unconvinced that crude analysis of numbers of positive studies is
helpful. Indeed, as the paper points out, the drug company funded studies
were of equal or better methodological quality than the non-funded. I am
particularly concerned that as the phrase "Publish or perish" becomes
ever more popular that people are blinded to the intrinsic bias that
arises in such an environment
Competing interests:
I was involved in drug company funded research more than 10 years ago. Currently I must publish or perish.
Competing interests: No competing interests
Industry-funded German Women Cohort Predictably Shows Cancer Prevention by Oral Contraceptives
Dear Editor,
a recent publication by Lexchin et al.1 stated that the studies perused indicated "that researchers cannot predict results of trials in advance", when funding by pharmaceutical firms had been achieved. Had any of the studies included covered the calendar year of 2002 and publications from Germany as well, a contrasting statement might have been resulted.
Results from the German Cohort Study on Women's Health have been published indicating that use of oral contraceptives are protective of breast cancer2 as well as female genital cancers3 (ovary, cervix, endometrium) . The study funded by Schering AG, Berlin, Germany, a major producer of hormone drugs, is a retrospective cohort of volunteer women, aged 18 - 65, recruited by advertisements, sickness fund flyers a.o.. Data are collected retrospectively from the birth of women onwards using a mailed questionnaire.
Initiated in 1998, 15.374 women contributed 610.328 women years until 2002. Prevalent cancers were reported by participants and had til publication not been validated by any means. Number of cases for two cancer locations were greatly below expected numbers, as determined from Saarland Cancer Registry incidence rates (women until age 65): ovarian cancer 24 observed (O) vs. 40.4 expected (E); breast cancer O: 308 vs. E: 420.
These results had to be expected taking into account case fatality rates of respective cancers. Thus cases of breast cancer in younger women which tend to be more aggressive and of poorer prognosis are to be underrepresented as well as more advanced cases of ovarian cancer, as these women had to be already deceased when recruitment started.
For multivariate logistic regression exposure to oral contraceptives was censored with date of diagnosis in cases, but not in controls. These distortions of established epidemiologic procedures for cohort studies predictably lead to a major bias towards diminishing relative risk estimates.
1Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome quality: a systematic review. BMJ 2003; 326: 1167-1170.
2Heinemann LAJ, Lewis MA, Kühl-Habich D, Braendle W, Moehner S (on behalf of the German Cohort Study on Women's Health Research Group). The risk of breast tumors and lifetime history of oral contraceptive use. Geburtsh Frauenheilk 2002; 62: 750-757.
3Heinemann LAJ, Lewis MA, Kühl-Habich D, Braendle W, Garbe E, Moehner S. Lifetime history of oral contraceptive use and development of tumors of the uterus and ovary. Geburtsh Frauenheilk 2002; 62: 566-573.
Competing interests:
None declared
Competing interests: No competing interests