Rapid Responses to:

RESEARCH: Anders W Jørgensen, Jørgen Hilden, and Peter C Gøtzsche Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review BMJ 2006; 333: 782 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Balanced comparison mandatory for new drugs launches

Rajan TD   (10 October 2006)

A short ten-step-questionnaire about possible bias in medical research

Manfred Gogol   (13 October 2006)

Industry sponsered Drug trials : It is commerce masquerading as science ! Let us stop the abuse !

venkatesan sangareddi   (14 October 2006)

Has Cochrane really achieved its goals?

Marko Tostad   (14 October 2006)

Doctored meta-analyses are not the only way drug companies manipulate physician drug use.

Rekha N Pillai, Manjith Narayanan, SpR, Paediatrics, Leicester.   (16 October 2006)

Cochrane bias

Stephen J Senn   (16 October 2006)

Before We Put Too Much Stock In Cochrane Systematic Reviews…Or Anyone’s…

James C. Coyne, none   (18 October 2006)

Journals must share the blame - word restrictions, not the funder, best explain these findings

Jonathan J Deeks   (19 October 2006)

Cochrane bias article

Douglas Keller   (23 October 2006)

Pharmaceutical Lies

Deepak Malhotra, 2000   (28 October 2006)

Inherent biases in publication process key confounders

Kostas Zacharias, Geetha Balakrishnan, Ameet Bakhai   (28 October 2006)

Cochrane reviews and other meta-analyses, authors’ reply

Anders W Jørgensen, Jørgen Hilden, Peter C Gøtzsche   (5 November 2006)

Introduce Evidence Based Medicine into Medical Education Curriculum

Babatunde Adetunji, MD, MA, MS, FASAM, Maju Mathews MD,MRCPsych: Director of Evidence based Medicine Course, Drexel University College of Medicine, Philadelphia. USA   (30 November 2006)

Balanced comparison mandatory for new drugs launches 10 October 2006
Rajan TD, Consultant,, CMPH Med College, Mumbai Specialist, Skin & Sex Transm Diseses, Tel: 0091-22-66982747 Send response to journal: Re: Balanced comparison mandatory for new drugs launches	The comparison of drug trials published by the Cochrane Collaboration and those supported by the pharmaceutical industry has arrived at a conclusion that few independent observers would disagree with. It is normal human tendency to scream," My baby is the prettiest!" Likewise, it is only natural for manufacturers who discover a new research molecule to make sure that it is widely accepted and consequently, a profit-making proposition. After all, if profits do not accrue, how can they persist with research, is an age-old argument. Newer molecules are often compared for efficacy against a placebo. While this is acceptable for those conditions where there are no preexisting effective drugs, the same cannot be said in disorders where other proven drugs exist. In cases where a new molecule is introduced for a disease which has comparative lines of therapy, the efficacy of the new molecule vis-a-vis the pre-existing drug should also be carried out before launching the drug. While this may become an expensive exercise for the pharmaceutical manufacturer, it conveys a clearer idea to the prescribing physician. In addition, the newer molecule is often steeply priced and in such a situation the physician has a better conviction to choose the newer drug if it is reported to be better, safer and stronger than the old drug. Competing interests: None declared
A short ten-step-questionnaire about possible bias in medical research 13 October 2006
Manfred Gogol, Head, Department of Geriatrics Krankenhaus Lindenbrunn, Lindenbrunn 1, Coppenbruegge 31863, Germany Send response to journal: Re: A short ten-step-questionnaire about possible bias in medical research	TO THE EDITOR: The report by JØrgensen et al. (1) adds relevant knowledge to the more and more recognized fact that lots of published research may be biased. Another ´good´ example of manipulation is the report by Melander et al (2). Other reports of fraud and misconduct (3) and also the possible influence on journals and scientific societies (4,5) are frightening. It doesn´t seem that the recommendations about our relations to the industry (6,7) will work . The problem remains that we have to face a mass of new data published every day with the necessity for clinical judegement. It can be unfair making a negative judgement only by missing a declaration of competing interests in general or existing financial relationships to a sponsor or the industry in general (8). But whom can we trust? A practical approach may be the following questionnaire with ten items adressing formal criteria of a paper. The questions are the result of my experience as a hospital physician. Maybe systematic research will add or remove some item from the questionnaire. 1. Is the research conducted by or together with the sponsor? 2. Is there an open access to the complete study design? 3. Is there a declaration of competing interests? 4. Are member of the publishing committee receiving money or other benefits from the sponsor (i.e. grants, consulting fees, lecture fees, other) and/or reporting stock holding? 5. Are members of the publishing committee employees and/or stock holders of the sponsor? 6. Are employees of the sponsor involved in any way in the data management? (If No. 5 is judged Yes or Unknown tick also the same here.) 7. Are employees of the sponsor involved in any way on judgement committees, i.e. clinical endpoint committee? (If No. 5 is judged Yes or Unknown tick also the same here.) 8. Was the study monitoring partly or totally done by the sponsor? 9. Was the data management partly or totally done by the sponsor or a for-profit organisation? 10. Does the abstract adress critically the limitations of the study? Judge each question with “yes”, “unknown” or “no”. Add up the answers and count two “unknown” as one “yes”. The possibility of bias arises with the number of questions you judged yes or unknown. Be careful if you have ticked yes 50% or more and discuss the paper with your colleagues. References: 1. JØrgensen AW, Hilden J, GØtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic reviews. BMJ 2006:332:782-6. 2. Melander H, Ahlqvist-Rastad J, Meifer G, Beermann B. Evidence b(i)ased medicine – selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug application. BMJ 2003;326:1171-6. 3. Martinson BC, Anderson MS, De Vries R. Scientists behaving badly. Nature 2005;435:737-8. 4. Lexchin J, Light DW. Commercial influence and the content of medical journals. BMJ 2006;332:1444-7. 5. Glassman PA, Hunter-Hayes J, Nakamura T. Pharmaceutical advertising revenue and physician organisations: how much is too much? WMJ 1999;171:234-8. 6. Coyle SL. Physician-industry relations. Part 1: Individual physicians. Ann Int Med 2002;136:396-402. 7. Coyle SL. Physician-industry relations. Part 2: Organizational issues. Ann Int Med 2002;136:403-6. 8. Dealing with disclosure. Nature Med 2006;12:979. Financial disclosure: I Have no financial support, holdings or patents of any kind related to drug or medical advice companies. Author contribution: I conceived and wrote this letter by myself. Sponsor´s role: No sponsor exists for the letter and my person. Competing interests: None declared
Industry sponsered Drug trials : It is commerce masquerading as science ! Let us stop the abuse ! 14 October 2006
venkatesan sangareddi, Assistant professor of cardiology Madras medical college.Chennai .India Send response to journal: Re: Industry sponsered Drug trials : It is commerce masquerading as science ! Let us stop the abuse !	Sir, This is in reference to the article Anders W Jørgensen . Any body following the modern day drug research will agree that, it is not a surprise conclusion ! Having conducted number of journal reviews as Assistant professor in a teaching hospital I can say with conviction pharma industry is notorious for their premature conclusions and hiding vital information . A study is generally continued till a positive outcome is obtained or synthesised . Generally “ It is the commerce that masquerades as science” and continue to play havoc in the population” What is more worrisome is that every body knows this fact. Is it not unfortunate a premier journal like BMJ has to remind the medical fraternity About the dangers and health hazards of health care industry.? The authors in fact are more polite in calling for a “Simple Caution statement”. In science we need not be polite we need to be only right. There have been number of instances the new drugs and devices are introduced prematurely and subsequently with drawn by the same companies .The concerned pharma firm sees to that the sensitive information are leaked out only after making enough profit out of the product .By that time enough damages is done. It is an irony there are examples where in even after a company bans a drug in one country continues to market in other developing countries. The other funny thing that is happening in leading medical journals is the mandatory declaration of conflicts and disclosure of author affiliations to the industry. One would wonder their purpose? Is it not fair to ask for a more serious caution notice In every article ? Akin to warning logo appearing against tobacco products ! “This article could be biased and applying the conclusion of this article in your day to day practice can be injurious and harmful to the well being of your patients”. ( This caution should appear Ideally just beneath the main Title ! ) Every one in the medical profession should be ashamed for clinical drug research having reached this stage. It is clearly a case of patient abuse in the name of science. It is an irony the so called guardians of health in the developing world is a silent spectator to this. Our ancestors suffered due to lack of scientific medicine. Modern man suffers because of it. The greatest threat to mankind is not from ignorance . It is from abuse of knowledge. We should know in the name of science we have absolutely no right to harm the people of our planet. Dr.S.Venkatesan Assistant professor of cardiology Madras Medical College Chennai. India Competing interests: None declared
Has Cochrane really achieved its goals? 14 October 2006
Marko Tostad, Epidemiologist Mt. Sinai 10029 Send response to journal: Re: Has Cochrane really achieved its goals?	Editors I am somewhat amazed that the BMJ chose to publish this review, given its small sample size yet broad conclusions. I don't believe it would be a stretch to suggest that if the results had been in the other direction, it would be less likely to have been published. It is true that Cochrane reviews report specific items more thoroughly than journal based reviews. However, much of this is due to the insistence of addressing methodological issues which are specious at times and the fact that Cochrane reviews are not limited by page length. For example, the issue of reporting allocation concealment, while it makes sense, does not mean that it was not done (1), nor does it even consistently demonstrate that it is an important methodological issue to report.(2) It is disappointing that the Cochrane library has become an ivory tower, given that many of the reviews are out of date and methodologically weak. The Cochrane library was established to be a clinically useful resource, but is that really true? There are many Cochrane reviews that would not be published in a paper journal, as they contain zero or just a few trials. There are far too many Cochrane reviews stating that, although upwards of 10 trials were found, the reporting is poor and therefore more research is required before a clinical recommendation can be made. Do you really think it is useful to say that several trials do not permit an inference on effectiveness? In closing, I would say that attacking pharma is an obvious target and one that amounts to little more than bullying. Pharma has an obvious conflict in wanting to publish favorable results. Why do the Cochrane group not go after the agencies claiming to promote health for the goodness of all, but mismanaging money and misusing evidence, such as the World Bank or WHO (3,4)- not targets that are so uniformly accepting of criticism. 1) Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM, Schunemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ, Guyatt GH. An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004 Dec;57(12):1232-6. 2) Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J. Correlation of quality measures with estimates of treatment effect in meta -analyses of randomized controlled trials. JAMA. 2002 Jun 12;287(22):2973-82. 3) Attaran A, Barnes KI, Bate R, Binka F, d'Alessandro U, Fanello CI, Garrett L, Mutabingwa TK, Roberts D, Sibley CH, Talisuna A, Van Geertruyden JP, Watkins WM. The World Bank: false financial and statistical accounts and medical malpractice in malaria treatment. 4) Attaran A, Barnes KI, Curtis C, d'Alessandro U, Fanello CI, Galinski MR, Kokwaro G, Looareesuwan S, Makanga M, Mutabingwa TK, Talisuna A, Trape JF, Watkins WM. WHO, the Global Fund, and medical malpractice in malaria treatment. Lancet. 2004 Jan 17;363(9404):237-40 Lancet. 2006 Jul 15;368(9531):247-52. Competing interests: None declared
Doctored meta-analyses are not the only way drug companies manipulate physician drug use. 16 October 2006
Rekha N Pillai, Foundation year 2 doctor Diana, Princess of Wales Hospital, Grimsby DN33 2BA, Manjith Narayanan, SpR, Paediatrics, Leicester. Send response to journal: Re: Doctored meta-analyses are not the only way drug companies manipulate physician drug use.	Dear Editor, The study by Jorgensen et al(1) is an eye opener to the data manipulations used by the drug companies to sell their products. The value of meta-analysis is severely hampered if data from all trials available are not used, or if data not available to the public is used. This has happened in most of the cases mentioned in the study concerned(1). However, we hold that this is not the only way drug companies try to manipulate physician-prescribing to their benefit. Selective highlighting of the trials that show their product in good light is a very common method known to doctors that have attended drug company sponsored events. It requires a very astute clinician who is up-to -date with his reading to fail to be impressed by the data presented. Publication bias is another important pitfall. Data that is available to the public and to the physicians is only those data the authors choose to send to the journals, the drug company chooses to fund or the journals choose to publish. Unfortunately, negative studies are not considered an advance by many researchers and much of the negative data are never published. This automatically skews all meta-analysis in favour of a new therapy. It is even more a cause for concern when authors choose to ignore or manipulate data that shows a new drug in bad light as happened with the Rofecoxib trial(2,3). The financial vested interests of the authors in the concerned drug company has been reported specifically(4). It is a great cause for concern that most of the new drugs reported in Jorgensen's study(1) that have shown equivocal data in Cochrane reviews are now quite commonly, if not exclusively, used for the disease in question. We suggest a few methods to avoid future bias in favour of new treatments. A central website which is open access could be maintained where authors are free to report any negative trials. Until a climate develops where negative trials are encouraged and reported without any publication bias, this seems to be the only option. Physicians have an obligation to know in detail about the trial before they change their prescribing habits. To this end, pharmaceutical companies should be encouraged to provide full reprints of the trial rather than just highlight the few graphs or tables that show their product in good stead. Reference: 1. JØrgensen AW, Hilden J, GØtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic reviews. BMJ 2006:332:782-6. 2. Curfman GD, Morrissey S, Drazen JM. Expression of Concern: Bombardier et al., "Comparison of Upper Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with Rheumatoid Arthritis," N Engl J Med 2000;343:1520-8. N Engl J Med; 353: 2813-4. 3. Bombardier C, Laine L, Reicin A, Shapiro D, Burgos-Vargas R, Davis B, Day R et al. Comparison of Upper Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with Rheumatoid Arthritis. N Engl J Med 2000;343(21):1520-8. 4. Supplement to: Bombardier C et al. Comparison of Upper Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with Rheumatoid Arthritis. N Engl J Med 2000;343(21):1520-8. available online http://content.nejm.org/cgi/content/full/343/21/1520/DC1 Competing interests: None declared
Cochrane bias 16 October 2006
Stephen J Senn, Professor of Statistics Department of Statistics, University of Glasgow, Glasgow, G12 9LL Send response to journal: Re: Cochrane bias	Once again the British Medical Journal treats its readers to the unedifying spectacle of Cochrane Collaboration (CC) members patting themselves on the back in public. Using the Oxman and Guyatt (O&G) index, which is the CC's favourite quality assessment tool, was devised by researchers at McMaster and 'validated' by asking colleagues at McMaster if they thought it was a good idea, some CC members 'evaluate' CC meta- analyses and find that they do rather well. What a surprise. Amongst important matters that the O&G index does not cover is 1. Was double counting avoided? 2. Were values used genuine or imputed? 3. If trials were of different design (e.g. parallel, cross-over or cluster- randomised) were they combined using appropriate methods? Given that the CC continue to insist on the O&G index as an appropriate way of judging the adequacy of meta-analysis, it is not surprising that they continue to produce analyses that fail to satisfy these three important criteria. Readers who want to try their hand at spotting these errors might like to look at the paper by Brockelbank et al in this journal and ask themselves if the O&G index helps them find them. Personnally I would like to see the Cochrane Collaboration (which is generally a force for good) spend more time thinking about analysis and less time boasting in public. Reference 1. Brocklebank D, Wright J, Cates C (2001) Systematic review of clinical effectiveness of pressurised metered dose inhalers versus other hand held inhaler devices for delivering corticosteroids in asthma. British Medical Journal 323:896-902 Competing interests: The author consults for the pharmaceutical industry and is an academic whose career is furthered by publishing
Before We Put Too Much Stock In Cochrane Systematic Reviews…Or Anyone’s… 18 October 2006
James C. Coyne, Professor of Psychology in Psychiatry University of Pennsylvania School of Medicine, Philadelphia, PA 19104, none Send response to journal: Re: Before We Put Too Much Stock In Cochrane Systematic Reviews…Or Anyone’s…	The demonstration by JØrgensen et al. (1) that industry sponsored meta-analyses differ in their conclusions and recommendations from non- industry sponsored meta-analyses should surprise no one. Yet, this demonstration should not lull us into believing that industry sponsorship is the only source of bias or that the Cochrane reviews to which the industry sponsored ones were compared should be in all cases uncritically accepted. It is well established that investigator allegiance is a strong determinant of the outcome of the evaluation of interventions, even in the absence of industry ties (2). Allegiance of the authors of meta-analyses have also been associated with selective attention to relevant studies, less critical evaluation of favorable studies, and more positive conclusions (3,4). This suggests a general need to scrutinize the backgrounds of authors, not just their declarations of conflict of interest. It also invites skepticism about a meta-analysis co-authored by the director of Cochrane Centre that puts the centre in such a favorable light. Cochrane reviews sometimes are conducted for literatures that are not yet ready for meta-analysis. As a case in point, a recent Cochrane meta- analysis concluded that there that couples therapy was not significantly better than individual therapy for depression (5). Whether couples therapy is offered should be a matter of “patient preference and availability of specific resources.” Such a conclusion is unlikely to prompt the redistribution of scarce resources to having marital therapists trained and available and may serve to discourage commitment of resources to an adequate comparison between the two forms of therapy because the authoritative Cochrane Collaboration has already spoken. Lack of evidence can easily be confused as evidence of a lack of differences. Yet, the studies reviewed were all seriously flawed, and none had close to the minimal cell size deemed necessary for inclusion in a meta-analysis (6), much less for a noninferiority or equivalence trial ( 7). Sometimes, the most appropriate conclusion is that a literature is not ready for meta- analysis (8. The Cochrane Collaborative has a tendency to err in the other direction, and with important implications for clinical practice and public policy. Whether the output of the Cochrane Collaborative is better, less biased, or less tainted by conflict of interest than reviews from other sources should be evaluated by someone other than a member of the collaborative. One of the founding members of the collaborative who still lists the collaborative as his institutional affiliation sometimes discloses his activities as an expert witness in product liability suits which his articles appear to benefit (9) and sometimes he fails to do so (10, 11). More to the point, Bjordal and colleagues (12) recently did a troubling analysis of a Cochrane report on low level laser therapy in osteoarthritis. They showed that only investigators who had performed trials with negative findings had been recruited to the review group. Furthermore, when asessed with a standard checklist for evaluating systematic reviews, the Cochrane review was found to be deficient in terms of trials not being included and omission of data, including subgroup analyses. Deficiencies were consistently in the direction of supporting the negative conclusions of the review. More such critical scrutiny of the output of Cochrane Collaborative is thus in order. At its website, the Cochrane Collaborative modestly describes itself as “the gold standard in evidence-based healthcare.” (http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME) In a headline in the This Week in BMJ accompanying the JØrgensen et al. paper admonished us to “Read industry supported drug reviews with caution.” This reasonable caution should be expanded to all reviews, including those of the Cochrane Collaborative. 1. JØrgensen AW, Hilden J, GØtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic reviews. BMJ 2006:332:782-6. 2. Luborsky L, Diguer L, Seligman DA, Rosenthal R, Krause ED, Johnson S, Halperin G, Bishop M, Berman JS, Schweizer E.The researcher's own therapy allegiances: A "wild card" in comparisons of treatment efficacy Clinical Psychology-Science and Practice. 1999 6 (1): 95-106. 3. Klein DF Flawed meta-analyses comparing psychotherapy with pharmacotherapy.American Journal of Psychiatry, 2000. 157 (8): 1204-1211. 4. Parker G, Roy K, Eyers K Cognitive behavior therapy for depression? Choose horses for courses American Journal of Psychiatry. 2003. 160 (5): 825-834. 5. Barbato A, D'Avanzo B. Marital therapy for depression. Cochrane Database Of Systematic Reviews. 20006 (2): Art. No. CD004188.pub2. 6. Kraemer, H. C., Gardner, C., Brooks, J. O., & Yesavage, J. A. Advantages of excluding underpowered studies in meta-analysis: Inclusionists versus exclusionists viewpoints. Psychological Methods,1998. 3, 23-31. 7. Le Henanff A, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials JAMA-Journal of the American Medical Association. 2006. 295(10): 1147-1151 8. Egger M, Smith GD, Phillips AN Meta-analysis: Principles and procedures British Medical Journal. 1997. 315 (7121): 1533-1537. 9. Herxheimer A, Healy, D., Menkes,D.B. Antidepressants and Violence: Problems at the Interface of Medicine and Law. PLoS Medicine 2006. 3, (9), DOI: 10.1371/journal.pmed.0030372 10. Herxheimer A, Mintzes B Antidepressants and adverse effects in young patients: uncovering the evidence. Canadian Medical Association Journal. 2004. 170 (4): 487-489. 11.Herxheimer A, Mintzes B. SSRI treatment for under-18s - Response. Canadian Medical Association Journal.2004. 170 (12): 1771-1772. 12. Bjordal JM, Lopes-Martins RAB, Klovning A Is quality control of Cochrane reviews in controversial areas sufficient? Journal of Alternative and Complementary Medicine. 2006. 12 (2): 181-183. Competing interests: None declared
Journals must share the blame - word restrictions, not the funder, best explain these findings 19 October 2006
Jonathan J Deeks, Professor of Health Statistics Department of Public Health and Epidemiology, University of Birmingham, Birmingham B15 2TT Send response to journal: Re: Journals must share the blame - word restrictions, not the funder, best explain these findings	Dear Editor Whilst there is little doubt that some industry funded systematic reviews use poor methodology and may misrepresent their findings, Jørgensen and colleagues’ study has overestimated this potential bias and misattributed differences in methodology and reporting to it [1]. First, the median quality score for the Cochrane reviews assessed was 7, whereas the scores for industry funding, undeclared funding and non- profit/no funding journal articles were 2, 2 and 3 respectively. These results are best explained by whether the review was published in The Cochrane Library or in another journal, not by the source of financial support. This links to word limits, which restrict reporting of methodological detail, and apply to journal articles but not Cochrane reviews. For example, the paper BMJ version of the industry funded review of celecoxib for RA in their sample had 2211 words [2], the online BMJ version had 3425 words, whilst the matching Cochrane review has 6002 words [3]. Second, at least for the celecoxib reviews, the reliability of unblinded quality assessments raises concern. For example, for assessment of allocation concealment, the Cochrane review states “concealment allocation was assessed and rated as A (blind randomisation), B (unclear methods of randomisation), or C (quasi-randomisation)”. The industry funded review stated “to assess the potential for bias we considered the method of randomisation, concealment of allocation …”. The first was deemed adequate by Jørgensen and colleagues, the second not. It is not clear why – both say that this assessment was undertaken but neither define adequate concealment. Similarly, Jørgensen and colleagues state “no industry reviews made reservations about their recommendations”. This is not consistent with the discussion section of the BMJ celecoxib review where 1) the importance of assessing serious GI events rather than endoscopically detected ulcers, 2) the lack of data to reliably determine upper gastrointestinal safety beyond six months, 3) the urgent need to assess cardiovascular safety of COX 2 inhibitors, and 4) the need for more data to assess the impact of concomitant aspirin therapy, are all raised as issues of concern. Interestingly it was the BMJ editors who choose to delete paragraphs expressing reservations 1) and 2) when preparing the abridged version for the BMJ paper journal. Blinded assessment would be possible if the study were repeated and only included published articles constrained by the restrictions on length imposed by traditional journals. This would allow a fairer comparison between funding source and reporting to be made that controlled for these word limits. The BMJ celecoxib review was produced as an experimental collaboration between industry and a respected academic unit with an international track record in systematic review methodology and substantial involvement in The Cochrane Collaboration. Industry provided details of all trials undertaken, gave access to the full trial reports and some funding. The academic unit produced a protocol which was agreed by all, then undertook data extraction, data analysis and interpretation. The BMJ paper was produced by the academic unit through a clause included in the contract allowing freedom to publish, including presentation of all unpublished trials and their findings. One author from industry was included on the paper to recognize their contribution to the review, which did not include any rights of veto over its content. We believe that this collaboration produced a robust, transparent and methodologically sound review. Having access to full study reports avoided problems experienced by the Cochrane reviewers in extracting detail from abridged journal articles, and is a move which should be applauded by those that campaign for fuller access to information. This included providing the company report for the CLASS trial, which gave substantially more detail than the corresponding JAMA article [4]. Industry has rightly been criticized for its failures to provide access to all available data [5]. Now, on the basis of very weak evidence, it stands accused of bias and interference when it tries to assist academic work by providing these documents. This will not encourage campgains for greater openness. The assessment of the likelihood of bias in systematic reviews, including Cochrane reviews, should always be made from assessing the reports of methods and the completeness of results of the review, and not prejudices about the organisations from which they emanate - this is the ethos of EBM. The lesson from Jørgensen and colleagues’ paper may be more that the opportunity to explain methods in the detail afforded by the Cochrane review format should be available for all systematic reviews. [1] Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic reviews. BMJ 2006:332:782. [2] Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper gastrointestinal safety of celecoxib for treatment of osteoarthritis and rheumatoid arthritis: systematic review of randomised controlled trials BMJ 2002;325:619. [3] Garner SE, Fidan DD, Frankish RR, Judd MG, Shea BJ, Towheed TE, Tugwell P, Wells G. Celecoxib for rheumatoid arthritis. Cochrane Database of Systematic Reviews 2002, Issue 4. Art. No.: CD003831. DOI: 10.1002/14651858.CD003831. [4] Silverstein FE, Faich G, Goldstein JL, Simon LS, Pincus T, Whelton A, Makuch R, Eisen G, Agrawal NM, Stenson WF, Burr AM, Zhao WW, Kent JD, Lefkowith JB, Verburg KM, Geis GS. Gastrointestinal toxicity with celecoxib vs nonsteroidal anti-inflammatory drugs for osteoarthritis and rheumatoid arthritis: the CLASS study: A randomized controlled trial. Celecoxib Long-term Arthritis Safety Study. JAMA. 2000; 284:1247-55. [5] House of Commons Health Committee. The Influence of the Pharmaceutical Industry. Fourth Report of the Session 2004-2005. London: Stationery Office, 2005. Competing interests: JJD has contributed to The Cochrane Collaboration for over a decade, is currently a member of the Steering Group of the Cochrane Collaboration and Treasurer. He has co-authored 14 Cochrane reviews, and is the lead editor of the Statistical Section of the Cochrane Handbook. The views expressed here are his own and not necessarily those of The Cochrane Collaboration. His previous unit (Centre for Statistics in Medicine, Oxford) received funding from Pfizer and Searle to undertake the review mentioned in [2] and JJD received consultancy fees from Pfizer from 2001-2. Both his previous and current units have received royalties from the BMJ on sales of reprints of reference [2].
Cochrane bias article 23 October 2006
Douglas Keller, retired pharmaceutical rep Albuquerque, NM 87109 Send response to journal: Re: Cochrane bias article	Editor, This article and the attached responses brought back many experiences from my 32 years of working with clinicians in private practice as well as the teaching environment. Most physicians accepted me and my information as being highly biased. However, I often observed that those who devaue the contributions of the pharmaceutical industry were often using this bias issue to avoid changes to their own approach to medicine and/or their political philosophy. I found the most dangerous bias issue in medical science was the one demostrated by the individual who had come to the conclusion that he or she had achieved a state of no bias. I suspect that these same individuals will not be able to detect bias in the Cochrane artice. Competing interests: None declared
Pharmaceutical Lies 28 October 2006
Deepak Malhotra, General Practice Sydney,Australia, 2000 Send response to journal: Re: Pharmaceutical Lies	Jorgensen's article on Cochrane review brings us to a situation where the general public looses all faith in a profession that rests on faith. Our researchers are greedy and today's world of everyone possesing a large house and driving an expensive car has made researchers into factory workers. Senior Consultants go on lecture tours with an 'honorarium'which is more than an average doctor's one years earnings. We have doctors from all over the world who drop into Australia on a first class all expenses paid junket/trip telling us how great a particular medicine is. If you read the small print on a phramaceutical company flier you will find most references are 'on file' or have been presented at a mid night session of the Darfur Cardiologists Conference.As a medical director of a pharmaceutical company I learnt how to get articles published in Journals with one journal promising publication if we purchased 2000 reprints at $10 each.Such dishonesty in the medical profession is a slur on the name of all those who dedicate their lives for betterment of society.As a basketball player once said that "do not measure your life by what you achieved but measure it by how many lives you touched". Competing interests: None declared
Inherent biases in publication process key confounders 28 October 2006
Kostas Zacharias, Cardiology Fellow Barnet General Hospital, Wellhouse Lane, Barnet EN5 3DJ, Geetha Balakrishnan, Ameet Bakhai Send response to journal: Re: Inherent biases in publication process key confounders	We read with interest the article by Jorgensen and colleagues (1) and would like to suggest some additional points of consideration with respect to their conclusions that industry supported reviews are less transparent, had few reservations about trial method limitations and had more favourable conclusions. The following items need address: 1. All 7 industry sponsored reviews were published in peer reviewed journals with considerable restriction on space 2. All 7 Cochrane reviews were published after the industry sponsored reviews and in some cases 3 years after, allowing time for further critical analyses and further data to be incorporated as well as trial method issues to be determined in the public domain. The authors have made little attempt of trying to deal with this complex effect of this time delay difference between the two types of publications. 3. All clinical trials now are required to be registered - and where possible trial protocols made available for review so that the previous artificial lack of transparency due to limitation of publication portals and space available is removed (2). Such facilities were not available in the past. 4. Industry supported reviews attract a high calibre of statistical support sufficient to pass safely through a rigorous peer reviewed process often with a statistical review as a separate hoop. Certainly the impact factors of the 7 industry papers are high and would reflect a high degree of scrutiny. Having participated in both industry led and Cochrane reviews, we would state that the scrutiny of industry internal processes is severe and challenging - and analyses are frequently duplicated separately to validate the findings. 5. We were unable to determine which of the Cochrane reviews had full access to individual patient level data. Industry-supported meta-analyses commonly use individual patient level data and are therefore often more robust. The authors have not informed us of these differences between studies. We would therefore suggest that the conclusion of these authors also merit caution and should perhaps discuss further limitations of their exercise.. Finally however - our recommendations are that ideally the two processes of Cochrane reviews and industry supported analyses should merge - using independent statisticians and researchers to undertake their own analyses but with data provided by industry without industry funding. Such a collaborative process now somewhat exists under the health technology appraisals undertaken by NICE (3) and it's reviews are transparent with well documented methods. More importantly - recommendations are open to appraisal in the public domain before being finalised - providing further chance to address or remove biases. Despite this - there will always be limitations in meta-analyses which by design attempt to compare separate studies undertaken at different times, in different populations, under different treatment circumstances (given that the standard control treatment changes with time). Thus such analyses always merit caution and cannot replace well- conducted large randomised trials (4). 1. Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ. 2006 Oct 14;333(7572):782. 2. Tonks A. Registering clinical trials. BMJ 1999;319:1565-1568 3. www.nice.org.uk 4. Flather MD, Farkouh ME, Pogue JM, Yusuf S. Strengths and limitations of meta-analysis: larger studies may be more reliable. Control Clin Trials. 1997 Dec;18(6):568-79; discussion 661-6. Competing interests: All authors undertake research supported by industry and non-industry related organisations in the UK
Cochrane reviews and other meta-analyses, authors’ reply 5 November 2006
Anders W Jørgensen, Physician Nordic Cochrane Centre, Rigshospitalet, DK-2100 Copenhagen Ø, Denmark, Jørgen Hilden, Peter C Gøtzsche Send response to journal: Re: Cochrane reviews and other meta-analyses, authors’ reply	Tostad, Deeks and Zacharias are concerned about the impact of space restrictions on our findings. Firstly, we believe space restrictions should not be an excuse for omitting important details on the methods used, as it is the authors who decide what to report within any given space, and as many journals allow additional material on the web. Secondly, our research reflects what is available to the readers, and not what could have been available, and it is therefore valid from a pragmatic perspective. If relevant details are not reported, e.g. methods used to ensure adequate allocation concealment and blinding, the readers may be unable to make their own assessments and conclusions, which may be different from those of the authors. Thirdly, we found a number of other interesting differences between Cochrane reviews and other meta-analyses than those related to methods. Senn comments on the validated scale we used for assessing methodological quality. We agree that there will always be problems with using scales, and this is precisely why we also looked at individual items that are known to be important for reducing bias in trials and in reviews. Inclusion of the additional items Senn suggests would not have changed our findings. Deeks mentions that reservations were made in his industry supported review. That is correct, but the reservations were made in the body of the Discussion. There were no such reservations in the abstract or in the conclusion, neither in the short, nor in the long, webbased version of the review which was the one we assessed (1). We evaluated the abstract and the conclusion for all the reviews when we judged whether the conclusions were without reservations and believe this is most relevant thing to do, as most people read only the abstract. Zacharias mistakenly writes that all 7 Cochrane reviews were published after the industry supported reviews. We included 8 pairs and not 7, and took great care to obtain comparable pairs. Our criterion was that the time span from the publication of a paper-based review and the date of the most recent substantive amendment of a Cochrane review (since these reviews are regularly updated) should not differ more than two years. As we have reported, the matching was very succesful. The median publication year was 2000 both for the industry supported reviews and for the matched Cochrane reviews, and the median difference in number of included trials was zero (2). We agree with Tostad and Coyne that some Cochrane reviews are not of good quality, and we gave examples of this (2). We urge readers who find problems with Cochrane reviews to submit a comment to be published as part of the review. This is very easy to do. Use “Add/View Feedback” in the Index that appears to the left of each review. Such feedback is most welcome as we constantly try to improve the quality and relevance of our reviews. 1. Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper gastrointestinal safety of celecoxib for treatment of osteoarthritis and rheumatoid arthritis: systematic review of randomised controlled trials. BMJ 2002;325:619. 2. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. doi:10.1136/bmj.38973.444699.0B, Published 6 October 2006. Competing interests: AWJ and PCG are affiliated with the Nordic Cochrane Centre. The views expressed in this letter represent those of the authors and are not necessarily the views or the official policy of the Cochrane Collaboration.
Introduce Evidence Based Medicine into Medical Education Curriculum 30 November 2006
Babatunde Adetunji, MD, MA, MS, FASAM, Clinical Assistant Professor of Psychiatry & Co-Director of Evidence Based Medicine Course Dept. of Psychiatry, Drexel University College of Medicine, Philadelphia. USA, Maju Mathews MD,MRCPsych: Director of Evidence based Medicine Course, Drexel University College of Medicine, Philadelphia. USA Send response to journal: Re: Introduce Evidence Based Medicine into Medical Education Curriculum	Dear Editor, Despite most reputable journals requesting for declaration of interests from participants of research studies, it is still no surprise that industry supported systematic reviews showed more bias than cochrane reviews of the same drugs. (1). After all, he who pays the piper dictates the tune. Even though in the hierarchy of evidence, a well conducted systematic review of level I studies will provide the best evidence but, if conducted without consideration for the limitation of bias, such review could serve as a source of multiplicative errors from poorly designed component trials. (2) The main question therefore becomes how can we protect physicians from being influenced by such reviews? Can we solely rely on regulatory bodies? No. We believe the answer lies in arming every physician with the tool to critically evaluate research studies. This can be done by incorporating the tenets of critical appraisal and evidence based medicine into medical education curriculum especially during postgraduate residency training. The Royal College of Psychiatrists introduced this into its membership examination curriculum in 1999. As far as we know, the Department of Psychiatry, Drexel University College of Medicine in Philadelphia is one of only a few residency programs that have evidence based medicine as a core curriculum course during residency training in USA. We believe that more training institutions as well as the individual members of the American Board of Medical Specialties should introduce it as a compulsory component of training and certification examinations. Once physicians have this crucial ability to critically appraise studies, they would be more adept at grading evidences as level I quality or relegate it to level IV quality that is useful only if there is nothing better in the database. References 1) Jorgensen AW, Hilden J and Gotzsche PC: Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ, Oct 2006; 333: 7 2) Reviewing and Grading Evidence: The Guidelines manual. National Institute for Health and Clinical Excellence: 2006, 49-190. Competing interests: None declared