Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review
BMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38973.444699.0B (Published 12 October 2006) Cite this as: BMJ 2006;333:782
All rapid responses
Tostad, Deeks and Zacharias are concerned about the impact of space
restrictions on our findings. Firstly, we believe space restrictions
should not be an excuse for omitting important details on the methods
used, as it is the authors who decide what to report within any given
space, and as many journals allow additional material on the web.
Secondly, our research reflects what is available to the readers, and not
what could have been available, and it is therefore valid from a pragmatic
perspective. If relevant details are not reported, e.g. methods used to
ensure adequate allocation concealment and blinding, the readers may be
unable to make their own assessments and conclusions, which may be
different from those of the authors. Thirdly, we found a number of other
interesting differences between Cochrane reviews and other meta-analyses
than those related to methods.
Senn comments on the validated scale we used for assessing
methodological quality. We agree that there will always be problems with
using scales, and this is precisely why we also looked at individual items
that are known to be important for reducing bias in trials and in reviews.
Inclusion of the additional items Senn suggests would not have changed our
findings.
Deeks mentions that reservations were made in his industry supported
review. That is correct, but the reservations were made in the body of the
Discussion. There were no such reservations in the abstract or in the
conclusion, neither in the short, nor in the long, webbased version of the
review which was the one we assessed (1). We evaluated the abstract and
the conclusion for all the reviews when we judged whether the conclusions
were without reservations and believe this is most relevant thing to do,
as most people read only the abstract.
Zacharias mistakenly writes that all 7 Cochrane reviews were
published after the industry supported reviews. We included 8 pairs and
not 7, and took great care to obtain comparable pairs. Our criterion was
that the time span from the publication of a paper-based review and the
date of the most recent substantive amendment of a Cochrane review (since
these reviews are regularly updated) should not differ more than two
years. As we have reported, the matching was very succesful. The median
publication year was 2000 both for the industry supported reviews and for
the matched Cochrane reviews, and the median difference in number of
included trials was zero (2).
We agree with Tostad and Coyne that some Cochrane reviews are not of
good quality, and we gave examples of this (2). We urge readers who find
problems with Cochrane reviews to submit a comment to be published as part
of the review. This is very easy to do. Use “Add/View Feedback” in the
Index that appears to the left of each review. Such feedback is most
welcome as we constantly try to improve the quality and relevance of our
reviews.
1. Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper
gastrointestinal safety of celecoxib for treatment of osteoarthritis and
rheumatoid arthritis: systematic review of randomised controlled trials.
BMJ 2002;325:619.
2. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic review. doi:10.1136/bmj.38973.444699.0B, Published 6
October 2006.
Competing interests:
AWJ and PCG are affiliated with the Nordic Cochrane
Centre. The views expressed in this letter represent those of the authors
and are not necessarily the views or the official policy of the Cochrane Collaboration.
Competing interests: No competing interests
We read with interest the article by Jorgensen and colleagues (1) and
would like to suggest some additional points of consideration with respect
to their conclusions that industry supported reviews are less transparent,
had few reservations about trial method limitations and had more
favourable conclusions.
The following items need address:
1. All 7 industry sponsored reviews were published in peer reviewed
journals with considerable restriction on space
2. All 7 Cochrane reviews were published after the industry sponsored
reviews and in some cases 3 years after, allowing time for further
critical analyses and further data to be incorporated as well as trial
method issues to be determined in the public domain.
The authors have made little attempt of trying to deal with this complex
effect of this time delay difference between the two types of
publications.
3. All clinical trials now are required to be registered - and where
possible trial protocols made available for review so that the previous
artificial lack of transparency due to limitation of publication portals
and space available is removed (2). Such facilities were not available in
the past.
4. Industry supported reviews attract a high calibre of statistical
support sufficient to pass safely through a rigorous peer reviewed process
often with a statistical review as a separate hoop. Certainly the impact
factors of the 7 industry papers are high and would reflect a high degree
of scrutiny. Having participated in both industry led and Cochrane
reviews, we would state that the scrutiny of industry internal processes
is severe and challenging - and analyses are frequently duplicated
separately to validate the findings.
5. We were unable to determine which of the Cochrane reviews had full
access to individual patient level data. Industry-supported meta-analyses
commonly use individual patient level data and are therefore often more
robust. The authors have not informed us of these differences between
studies.
We would therefore suggest that the conclusion of these authors also
merit caution and should perhaps discuss further limitations of their
exercise..
Finally however - our recommendations are that ideally the two
processes of Cochrane reviews and industry supported analyses should merge
- using independent statisticians and researchers to undertake their own
analyses but with data provided by industry without industry funding. Such
a collaborative process now somewhat exists under the health technology
appraisals undertaken by NICE (3) and it's reviews are transparent with
well documented methods. More importantly - recommendations are open to
appraisal in the public domain before being finalised - providing further
chance to address or remove biases.
Despite this - there will always be limitations in meta-analyses
which by design attempt to compare separate studies undertaken at
different times, in different populations, under different treatment
circumstances (given that the standard control treatment changes with
time). Thus such analyses always merit caution and cannot replace well-
conducted large randomised trials (4).
1. Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic review. BMJ. 2006 Oct 14;333(7572):782.
2. Tonks A. Registering clinical trials. BMJ 1999;319:1565-1568
3. www.nice.org.uk
4. Flather MD, Farkouh ME, Pogue JM, Yusuf S. Strengths and limitations of
meta-analysis: larger studies may be more reliable. Control Clin Trials.
1997 Dec;18(6):568-79; discussion 661-6.
Competing interests:
All authors undertake research supported by industry and non-industry related organisations in the UK
Competing interests: No competing interests
Jorgensen's article on Cochrane review brings us to a situation where
the general public looses all faith in a profession that rests on faith.
Our researchers are greedy and today's world of everyone possesing a large house
and driving an expensive car has made researchers into factory workers.
Senior Consultants go on lecture tours with an 'honorarium'which is more
than an average doctor's one years earnings. We have doctors from all over
the world who drop into Australia on a first class all expenses paid
junket/trip telling us how great a particular medicine is. If you read the
small print on a phramaceutical company flier you will find most
references are 'on file' or have been presented at a mid night session of
the Darfur Cardiologists Conference.As a medical director of a
pharmaceutical company I learnt how to get articles published in Journals
with one journal promising publication if we purchased 2000
reprints at $10 each.Such dishonesty in the medical profession is a slur
on the name of all those who dedicate their lives for betterment of
society.As a basketball player once said that "do not measure your life by
what you achieved but measure it by how many lives you touched".
Competing interests:
None declared
Competing interests: No competing interests
Editor,
This article and the attached responses brought back many experiences
from my 32 years of working with clinicians in private practice as well as
the teaching environment.
Most physicians accepted me and my information as being highly
biased. However, I often observed that those who devaue the contributions
of the pharmaceutical industry were often using this bias issue to avoid
changes to their own approach to medicine and/or their political
philosophy.
I found the most dangerous bias issue in medical science was the one
demostrated by the individual who had come to the conclusion that he or
she had achieved a state of no bias. I suspect that these same individuals
will not be able to detect bias in the Cochrane artice.
Competing interests:
None declared
Competing interests: No competing interests
Dear Editor
Whilst there is little doubt that some industry funded systematic
reviews use poor methodology and may misrepresent their findings,
Jørgensen and colleagues’ study has overestimated this potential bias and
misattributed differences in methodology and reporting to it [1].
First, the median quality score for the Cochrane reviews assessed was
7, whereas the scores for industry funding, undeclared funding and non-
profit/no funding journal articles were 2, 2 and 3 respectively. These
results are best explained by whether the review was published in The
Cochrane Library or in another journal, not by the source of financial
support. This links to word limits, which restrict reporting of
methodological detail, and apply to journal articles but not Cochrane
reviews. For example, the paper BMJ version of the industry funded review
of celecoxib for RA in their sample had 2211 words [2], the online BMJ
version had 3425 words, whilst the matching Cochrane review has 6002 words
[3].
Second, at least for the celecoxib reviews, the reliability of
unblinded quality assessments raises concern. For example, for assessment
of allocation concealment, the Cochrane review states “concealment
allocation was assessed and rated as A (blind randomisation), B (unclear
methods of randomisation), or C (quasi-randomisation)”. The industry
funded review stated “to assess the potential for bias we considered the
method of randomisation, concealment of allocation …”. The first was
deemed adequate by Jørgensen and colleagues, the second not. It is not
clear why – both say that this assessment was undertaken but neither
define adequate concealment.
Similarly, Jørgensen and colleagues state “no industry reviews made
reservations about their recommendations”. This is not consistent with the
discussion section of the BMJ celecoxib review where 1) the importance of
assessing serious GI events rather than endoscopically detected ulcers, 2)
the lack of data to reliably determine upper gastrointestinal safety
beyond six months, 3) the urgent need to assess cardiovascular safety of
COX 2 inhibitors, and 4) the need for more data to assess the impact of
concomitant aspirin therapy, are all raised as issues of concern.
Interestingly it was the BMJ editors who choose to delete paragraphs
expressing reservations 1) and 2) when preparing the abridged version for
the BMJ paper journal.
Blinded assessment would be possible if the study were repeated and
only included published articles constrained by the restrictions on length
imposed by traditional journals. This would allow a fairer comparison
between funding source and reporting to be made that controlled for these
word limits.
The BMJ celecoxib review was produced as an experimental
collaboration between industry and a respected academic unit with an
international track record in systematic review methodology and
substantial involvement in The Cochrane Collaboration. Industry provided
details of all trials undertaken, gave access to the full trial reports
and some funding. The academic unit produced a protocol which was agreed
by all, then undertook data extraction, data analysis and interpretation.
The BMJ paper was produced by the academic unit through a clause included
in the contract allowing freedom to publish, including presentation of all
unpublished trials and their findings. One author from industry was
included on the paper to recognize their contribution to the review, which
did not include any rights of veto over its content. We believe that this
collaboration produced a robust, transparent and methodologically sound
review. Having access to full study reports avoided problems experienced
by the Cochrane reviewers in extracting detail from abridged journal
articles, and is a move which should be applauded by those that campaign
for fuller access to information. This included providing the company
report for the CLASS trial, which gave substantially more detail than the
corresponding JAMA article [4].
Industry has rightly been criticized for its failures to provide
access to all available data [5]. Now, on the basis of very weak
evidence, it stands accused of bias and interference when it tries to
assist academic work by providing these documents. This will not
encourage campgains for greater openness. The assessment of the likelihood
of bias in systematic reviews, including Cochrane reviews, should always
be made from assessing the reports of methods and the completeness of
results of the review, and not prejudices about the organisations from
which they emanate - this is the ethos of EBM. The lesson from Jørgensen
and colleagues’ paper may be more that the opportunity to explain methods
in the detail afforded by the Cochrane review format should be available
for all systematic reviews.
[1] Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic reviews. BMJ 2006:332:782.
[2] Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper
gastrointestinal safety of celecoxib for treatment of osteoarthritis and
rheumatoid arthritis: systematic review of randomised controlled trials
BMJ 2002;325:619.
[3] Garner SE, Fidan DD, Frankish RR, Judd MG, Shea BJ, Towheed TE,
Tugwell P, Wells G. Celecoxib for rheumatoid arthritis. Cochrane Database
of Systematic Reviews 2002, Issue 4. Art. No.: CD003831. DOI:
10.1002/14651858.CD003831.
[4] Silverstein FE, Faich G, Goldstein JL, Simon LS, Pincus T,
Whelton A, Makuch R, Eisen G, Agrawal NM, Stenson WF, Burr AM, Zhao WW,
Kent JD, Lefkowith JB, Verburg KM, Geis GS. Gastrointestinal toxicity with
celecoxib vs nonsteroidal anti-inflammatory drugs for osteoarthritis and
rheumatoid arthritis: the CLASS study: A randomized controlled trial.
Celecoxib Long-term Arthritis Safety Study. JAMA. 2000; 284:1247-55.
[5] House of Commons Health Committee. The Influence of the
Pharmaceutical Industry. Fourth Report of the Session 2004-2005. London:
Stationery Office, 2005.
Competing interests:
JJD has contributed to The Cochrane Collaboration for over a decade, is currently a member of the Steering Group of the Cochrane Collaboration and Treasurer. He has co-authored 14 Cochrane reviews, and is the lead editor of the Statistical Section of the Cochrane Handbook. The views expressed here are his own and not necessarily those of The Cochrane Collaboration. His previous unit (Centre for Statistics in Medicine, Oxford) received funding from Pfizer and Searle to undertake the review mentioned in [2] and JJD received consultancy fees from Pfizer from 2001-2. Both his previous and current units have received royalties from the BMJ on sales of reprints of reference [2].
Competing interests: No competing interests
The demonstration by JØrgensen et al. (1) that industry sponsored
meta-analyses differ in their conclusions and recommendations from non-
industry sponsored meta-analyses should surprise no one. Yet, this
demonstration should not lull us into believing that industry sponsorship
is the only source of bias or that the Cochrane reviews to which the
industry sponsored ones were compared should be in all cases uncritically
accepted.
It is well established that investigator allegiance is a strong
determinant of the outcome of the evaluation of interventions, even in the
absence of industry ties (2). Allegiance of the authors of meta-analyses
have also been associated with selective attention to relevant studies,
less critical evaluation of favorable studies, and more positive
conclusions (3,4). This suggests a general need to scrutinize the
backgrounds of authors, not just their declarations of conflict of
interest. It also invites skepticism about a meta-analysis co-authored by
the director of Cochrane Centre that puts the centre in such a favorable
light.
Cochrane reviews sometimes are conducted for literatures that are not
yet ready for meta-analysis. As a case in point, a recent Cochrane meta-
analysis concluded that there that couples therapy was not significantly
better than individual therapy for depression (5). Whether couples therapy
is offered should be a matter of “patient preference and availability of
specific resources.” Such a conclusion is unlikely to prompt the
redistribution of scarce resources to having marital therapists trained
and available and may serve to discourage commitment of resources to an
adequate comparison between the two forms of therapy because the
authoritative Cochrane Collaboration has already spoken. Lack of evidence
can easily be confused as evidence of a lack of differences. Yet, the
studies reviewed were all seriously flawed, and none had close to the
minimal cell size deemed necessary for inclusion in a meta-analysis (6),
much less for a noninferiority or equivalence trial ( 7). Sometimes, the
most appropriate conclusion is that a literature is not ready for meta-
analysis (8. The Cochrane Collaborative has a tendency to err in the other
direction, and with important implications for clinical practice and
public policy.
Whether the output of the Cochrane Collaborative is better, less
biased, or less tainted by conflict of interest than reviews from other
sources should be evaluated by someone other than a member of the
collaborative. One of the founding members of the collaborative who still
lists the collaborative as his institutional affiliation sometimes
discloses his activities as an expert witness in product liability suits
which his articles appear to benefit (9) and sometimes he fails to do so
(10, 11). More to the point, Bjordal and colleagues (12) recently did a
troubling analysis of a Cochrane report on low level laser therapy in
osteoarthritis. They showed that only investigators who had performed
trials with negative findings had been recruited to the review group.
Furthermore, when asessed with a standard checklist for evaluating
systematic reviews, the Cochrane review was found to be deficient in terms
of trials not being included and omission of data, including subgroup
analyses. Deficiencies were consistently in the direction of supporting
the negative conclusions of the review. More such critical scrutiny of the
output of Cochrane Collaborative is thus in order.
At its website, the Cochrane Collaborative modestly describes itself
as “the gold standard in evidence-based healthcare.”
(http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME) In a
headline in the This Week in BMJ accompanying the JØrgensen et al. paper
admonished us to “Read industry supported drug reviews with caution.”
This reasonable caution should be expanded to all reviews, including those
of the Cochrane Collaborative.
1. JØrgensen AW, Hilden J, GØtzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic reviews. BMJ 2006:332:782-6.
2. Luborsky L, Diguer L, Seligman DA, Rosenthal R, Krause ED, Johnson
S, Halperin G, Bishop M, Berman JS, Schweizer E.The researcher's own
therapy allegiances: A "wild card" in comparisons of treatment efficacy
Clinical Psychology-Science and Practice. 1999 6 (1): 95-106.
3. Klein DF Flawed meta-analyses comparing psychotherapy with
pharmacotherapy.American Journal of Psychiatry, 2000. 157 (8): 1204-1211.
4. Parker G, Roy K, Eyers K Cognitive behavior therapy for
depression? Choose horses for courses American Journal of Psychiatry.
2003. 160 (5): 825-834.
5. Barbato A, D'Avanzo B. Marital therapy for depression. Cochrane
Database Of Systematic Reviews. 20006 (2): Art. No. CD004188.pub2.
6. Kraemer, H. C., Gardner, C., Brooks, J. O., & Yesavage, J. A.
Advantages of excluding underpowered studies in meta-analysis:
Inclusionists versus exclusionists viewpoints. Psychological Methods,1998.
3, 23-31.
7. Le Henanff A, Giraudeau B, Baron G, Ravaud P. Quality of reporting
of noninferiority and equivalence randomized trials JAMA-Journal of the
American Medical Association. 2006. 295(10): 1147-1151
8. Egger M, Smith GD, Phillips AN Meta-analysis: Principles and
procedures British Medical Journal. 1997. 315 (7121): 1533-1537.
9. Herxheimer A, Healy, D., Menkes,D.B. Antidepressants and Violence:
Problems at the Interface of Medicine and Law. PLoS Medicine 2006. 3,
(9), DOI: 10.1371/journal.pmed.0030372
10. Herxheimer A, Mintzes B Antidepressants and adverse effects in
young patients: uncovering the evidence. Canadian Medical Association
Journal. 2004. 170 (4): 487-489.
11.Herxheimer A, Mintzes B. SSRI treatment for under-18s - Response.
Canadian Medical Association Journal.2004. 170 (12): 1771-1772.
12. Bjordal JM, Lopes-Martins RAB, Klovning A Is quality control of
Cochrane reviews in controversial areas sufficient? Journal of Alternative
and Complementary Medicine. 2006. 12 (2): 181-183.
Competing interests:
None declared
Competing interests: No competing interests
Once again the British Medical Journal treats its readers to the
unedifying spectacle of Cochrane Collaboration (CC) members patting
themselves on the back in public. Using the Oxman and Guyatt (O&G) index,
which is the CC's favourite quality assessment tool, was devised by
researchers at McMaster and 'validated' by asking colleagues at McMaster
if they thought it was a good idea, some CC members 'evaluate' CC meta-
analyses and find that they do rather well. What a surprise.
Amongst important matters that the O&G index does not cover is 1. Was
double counting avoided? 2. Were values used genuine or imputed? 3. If
trials were of different design (e.g. parallel, cross-over or cluster-
randomised) were they combined using appropriate methods?
Given that the CC continue to insist on the O&G index as an
appropriate way of judging the adequacy of meta-analysis, it is not
surprising that they continue to produce analyses that fail to satisfy
these three important criteria. Readers who want to try their hand at
spotting these errors might like to look at the paper by Brockelbank et al
in this journal and ask themselves if the O&G index helps them find them.
Personnally I would like to see the Cochrane Collaboration (which is
generally a force for good) spend more time thinking about analysis and
less time boasting in public.
Reference
1. Brocklebank D, Wright J, Cates C (2001) Systematic review of
clinical effectiveness of pressurised metered dose inhalers versus other
hand held inhaler devices for delivering corticosteroids in asthma.
British Medical Journal 323:896-902
Competing interests:
The author consults for the pharmaceutical industry and is an academic whose career is furthered by publishing
Competing interests: No competing interests
Dear Editor,
The study by Jorgensen et al(1) is an eye opener to the data
manipulations used by the drug companies to sell their products. The value
of meta-analysis is severely hampered if data from all trials available
are not used, or if data not available to the public is used. This has
happened in most of the cases mentioned in the study concerned(1).
However, we hold that this is not the only way drug companies try to
manipulate physician-prescribing to their benefit.
Selective highlighting of the trials that show their product in good
light is a very common method known to doctors that have attended drug
company sponsored events. It requires a very astute clinician who is up-to
-date with his reading to fail to be impressed by the data presented.
Publication bias is another important pitfall. Data that is available
to the public and to the physicians is only those data the authors choose
to send to the journals, the drug company chooses to fund or the journals
choose to publish. Unfortunately, negative studies are not considered an
advance by many researchers and much of the negative data are never
published. This automatically skews all meta-analysis in favour of a new
therapy.
It is even more a cause for concern when authors choose to ignore or
manipulate data that shows a new drug in bad light as happened with the
Rofecoxib trial(2,3). The financial vested interests of the authors in the
concerned drug company has been reported specifically(4).
It is a great cause for concern that most of the new drugs reported
in Jorgensen's study(1) that have shown equivocal data in Cochrane reviews
are now quite commonly, if not exclusively, used for the disease in
question.
We suggest a few methods to avoid future bias in favour of new
treatments. A central website which is open access could be maintained
where authors are free to report any negative trials. Until a climate
develops where negative trials are encouraged and reported without any
publication bias, this seems to be the only option.
Physicians have an obligation to know in detail about the trial
before they change their prescribing habits. To this end, pharmaceutical
companies should be encouraged to provide full reprints of the trial
rather than just highlight the few graphs or tables that show their
product in good stead.
Reference:
1. JØrgensen AW, Hilden J, GØtzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic reviews. BMJ 2006:332:782-6.
2. Curfman GD, Morrissey S, Drazen JM. Expression of Concern:
Bombardier et al., "Comparison of Upper Gastrointestinal Toxicity of
Rofecoxib and Naproxen in Patients with Rheumatoid Arthritis," N Engl J
Med 2000;343:1520-8. N Engl J Med; 353: 2813-4.
3. Bombardier C, Laine L, Reicin A, Shapiro D, Burgos-Vargas R, Davis
B, Day R et al. Comparison of Upper Gastrointestinal Toxicity of Rofecoxib
and Naproxen in Patients with Rheumatoid Arthritis. N Engl J Med
2000;343(21):1520-8.
4. Supplement to: Bombardier C et al. Comparison of Upper
Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with
Rheumatoid Arthritis. N Engl J Med 2000;343(21):1520-8. available online
http://content.nejm.org/cgi/content/full/343/21/1520/DC1
Competing interests:
None declared
Competing interests: No competing interests
Editors
I am somewhat amazed that the BMJ chose to publish this review, given
its small sample size yet broad conclusions. I don't believe it would be
a stretch to suggest that if the results had been in the other direction,
it would be less likely to have been published.
It is true that Cochrane reviews report specific items more
thoroughly than journal based reviews. However, much of this is due to the
insistence of addressing methodological issues which are specious at times
and the fact that Cochrane reviews are not limited by page length. For
example, the issue of reporting allocation concealment, while it makes
sense, does not mean that it was not done (1), nor does it even
consistently demonstrate that it is an important methodological issue to
report.(2)
It is disappointing that the Cochrane library has become an ivory
tower, given that many of the reviews are out of date and methodologically
weak. The Cochrane library was established to be a clinically useful
resource, but is that really true? There are many Cochrane reviews that
would not be published in a paper journal, as they contain zero or just a
few trials. There are far too many Cochrane reviews stating that, although
upwards of 10 trials were found, the reporting is poor and therefore more
research is required before a clinical recommendation can be made. Do you
really think it is useful to say that several trials do not permit an
inference on effectiveness?
In closing, I would say that attacking pharma is an obvious target
and one that amounts to little more than bullying. Pharma has an obvious
conflict in wanting to publish favorable results. Why do the Cochrane
group not go after the agencies claiming to promote health for the
goodness of all, but mismanaging money and misusing evidence, such as the
World Bank or WHO (3,4)- not targets that are so uniformly accepting of
criticism.
1) Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM,
Schunemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ,
Guyatt GH.
An observational study found that authors of randomized controlled trials
frequently use concealment of randomization and blinding, despite the
failure to report these methods. J Clin Epidemiol. 2004 Dec;57(12):1232-6.
2) Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C,
Lau J.
Correlation of quality measures with estimates of treatment effect in meta
-analyses of randomized controlled trials.
JAMA. 2002 Jun 12;287(22):2973-82.
3) Attaran A, Barnes KI, Bate R, Binka F, d'Alessandro U, Fanello CI,
Garrett L, Mutabingwa TK, Roberts D, Sibley CH, Talisuna A, Van
Geertruyden JP, Watkins WM. The World Bank: false financial and
statistical accounts and medical malpractice in malaria treatment.
4) Attaran A, Barnes KI, Curtis C, d'Alessandro U, Fanello CI,
Galinski MR, Kokwaro G, Looareesuwan S, Makanga M, Mutabingwa TK, Talisuna
A, Trape JF, Watkins WM.
WHO, the Global Fund, and medical malpractice in malaria treatment.
Lancet. 2004 Jan 17;363(9404):237-40
Lancet. 2006 Jul 15;368(9531):247-52.
Competing interests:
None declared
Competing interests: No competing interests
Introduce Evidence Based Medicine into Medical Education Curriculum
Dear Editor,
Despite most reputable journals requesting for declaration of
interests from participants of research studies, it is still no surprise
that industry supported systematic reviews showed more bias than cochrane
reviews of the same drugs. (1). After all, he who pays the piper dictates
the tune. Even though in the hierarchy of evidence, a well conducted
systematic review of level I studies will provide the best evidence but,
if conducted without consideration for the limitation of bias, such review
could serve as a source of multiplicative errors from poorly designed
component trials. (2)
The main question therefore becomes how can we protect physicians
from being influenced by such reviews? Can we solely rely on regulatory
bodies? No. We believe the answer lies in arming every physician with the
tool to critically evaluate research studies. This can be done by
incorporating the tenets of critical appraisal and evidence based medicine
into medical education curriculum especially during postgraduate residency
training. The Royal College of Psychiatrists introduced this into its
membership examination curriculum in 1999. As far as we know, the
Department of Psychiatry, Drexel University College of Medicine in
Philadelphia is one of only a few residency programs that have evidence
based medicine as a core curriculum course during residency training in
USA. We believe that more training institutions as well as the individual
members of the American Board of Medical Specialties should introduce it
as a compulsory component of training and certification examinations.
Once physicians have this crucial ability to critically appraise
studies, they would be more adept at grading evidences as level I quality
or relegate it to level IV quality that is useful only if there is nothing
better in the database.
References
1) Jorgensen AW, Hilden J and Gotzsche PC: Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic review. BMJ, Oct 2006; 333: 7
2) Reviewing and Grading Evidence: The Guidelines manual. National
Institute for Health and Clinical Excellence: 2006, 49-190.
Competing interests:
None declared
Competing interests: No competing interests