Rapid Responses to:

RESEARCH: Nikolaos A Patsopoulos, John P A Ioannidis, and Apostolos A Analatos Origin and funding of the most frequently cited papers in medicine: database analysis BMJ 2006; 332: 1061-1064 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Incomplete reasoning

James Michelson   (6 May 2006)

Re: Incomplete reasoning

Nikolaos A Patsopoulos, Apostolos A Analatos, John PA Ioannidis   (26 May 2006)

The most frequently cited scientific paper in health science?

Flemming M Nilsson   (12 November 2006)

On citing scientific papers properly

Reinhard Wentz   (13 November 2006)

Incomplete reasoning 6 May 2006
James Michelson, Prof. Orthop. Surg., Dir Clinical Informatics George Washington Univ., USA, 20037 Send response to journal: Re: Incomplete reasoning	While I applaud the authors of this study on exploring an important facet of how medical research is getting funded, the conclusions they draw are overly broad, speculative, and not actually supported by their data. They seem to have confused association with causation as they state that the increased funding of research by industry is determining the research agenda. Although that is possible, it is also possible that exciting new research avenues that are being opened by academic researchers is attracting the funding, rather than the reverse. A good example of this is the now-frequent transition of academic researchers from academia to entrepreneurs with their new discoveries. This is not a value judgment as to whether this is a good thing, but it does not support the contention of the authors that “Clinical research is dictated by the need to promote products of industry”. There is also the implication by the authors that the research results reported in industry-funded studies is more likely to show positive effects, and they quote an earlier study supporting this. However, the authors actually collected the data necessary to examine this question yet did not mention that aspect of their findings in their own Results. I can only assume that they did not find such an association between funding sources and study results. If this is true, then the authors are remiss in not explicitly reporting on this. Finally, the authors do a great disservice to the leading medical journals by speculating that they publish articles that are more likely to be “popular” than meticulous and unbiased. If anything, the leading journals have gone out of their way to proclaim that they will not follow such a path, precisely for the reasons cited in the current study. I am disappointed in the alarmist tone of this article. Despite addressing a serious concern in academic medicine, its unsupported speculations and allegations detract from what should be a rigorous discussion. Competing interests: None declared
Re: Incomplete reasoning 26 May 2006
Nikolaos A Patsopoulos, Research associate Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, Apostolos A Analatos, John PA Ioannidis Send response to journal: Re: Re: Incomplete reasoning	We are afraid that Dr. Michelson’s comments reflect mostly incomplete reading of our paper and the accompanying appendix. The claim that perhaps “exciting new research avenues that are being opened by academic researchers is attracting the [industry] funding” is unlikely to be true. Simply looking at the type of interventions of the 77 highly-cited randomized trials, almost all of them examined interventions largely developed by the industry. Only two trials examined purely changes in lifestyle and ventilation with lower tidal volumes and another 3 looked into tighter control of disease with already established drug interventions or use of an effective intervention in an earlier stage of the disease. The remaining 94% were typical drug, supplement, or medical device trials – technologies that are rarely discovered and/or developed in university labs nowadays. We have also previously shown that the strongest predictor of a promising basic science finding moving into randomized trials is the involvement of industry authors in the basic research [1]. Our study did not aim to investigate specifically the association of industry funding and effectiveness of the experimental interventions. Our design is not optimally powered for that question. These are the most- cited trials, so one expects a preponderance of “positive” results and few “negative” results to contrast against. Indeed, amongst the 77 trials, there were 82 independent comparisons of interventions: 69 proved effective, while 13 proved ineffective or harmful. The percentage of industry funding was 87% and 69%, respectively, leading to an odds ratio of 2.96 (95% CI: 0.75-11.7), in full agreement with previous empirical investigations [2]. Finally, Dr. Michelson certainly misinterpreted our discussion about medical journals. Conversely, we stated that these journals want to promote excellence and to attract articles of better quality. This is common sense. The problem is how best to achieve and safeguard this attitude in the current research and funding environment. 1. Contopoulos-Ioannidis DG, Ntzani E, Ioannidis JP. Translation of highly promising basic science research into clinical applications. Am J Med 2003;114:477-84. 2. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003;326:1167-70. Competing interests: None declared
The most frequently cited scientific paper in health science? 12 November 2006
Flemming M Nilsson, MD, PhD,Consultant in old age psychiatry Copenhagen University Hospital, Amager Hospital, Digevej 110, DK-2300 Copenhagen, Denmark Send response to journal: Re: The most frequently cited scientific paper in health science?	EDITOR-Why are some papers cited again and again, while for others, you really wonder why nobody ever recognized it? Scientific papers with a high impact have been investigated in recent years [1]. The paper investigated in this letter [2] was destined to take a course of its very own. It is probably the most cited paper in the history of health science, and it is by far the most cited paper in neuroscience, old age psychiatry and geriatrics. The Mini Mental State Examination (MMSE) [2] is a brief, quantitative measure of cognitive status in adults. It has been used as a screening instrument to estimate the severity of cognitive impairment - at a given point of time, to follow the course of cognitive changes in an individual over time, and to document an individual’s response to treatment. The standard MMSE published in 1975 is still in use, with only minor subsequent modifications done by the authors. The MMSE has demonstrated validity and reliability in psychiatry, neurology, geriatrics, as well as in other medical specialities. The MMSE has been translated into more than 35 authorized foreign language editions. A bibliometric measurement was done in commercial databases provided by my university library. All citations of the paper between 1975 and 2006 were downloaded from Web of Knowledge (Institute for Scientific Information (ISI)). Analysis of the frequency of citations per year and distribution of specialities were done by the programmes provided in the Web of Knowledge. As of November 2006 there were found 19,243 citations on the article since 1977. Frequency of citations per year is shown in figure 1. This shows an exponential increase in the lifetime of the rating scale, which is more than thirty years. The distribution of specialities in frequency of citations is not surprisingly that clinical neurology (5900) and psychiatry (5700) and geriatrics (4500) are the most prominent with equally comparable numbers of citations. Clinical psychology and internal medicine are the succeeding specialities with approximately 1500 citations each. Many papers apply to several specialities. The Mini Mental State Examination (MMSE) is one of the most ever cited research papers in medical science. Conclusion: The paper by Marshal Folstein and colleagues is a very highly cited paper and probably the most frequently cited ever in medicine. It will most likely be that for many years to come. Just to add prosperity to this prophecy, I quote it here once more [2]. I tried to explore if there were reasons for the fate of this paper. Certain elements are obvious, but that it was merely pure chance was not found to be the case. More likely the explanation is that you have to be cited in the right place, in the right time, and probably also by the right people. The “across speciality” use of the scale explains a great part of its success. Recommendations for further research in this field requires that there will be highly cited papers in the future. Young scientist should therefore consider to create (or get!) an idea for a very simple questionnaire or manoeuvre. Make it brief. Make it useful. Give it an appealing name, with implication of shortness, quickness, or simply that it will make your life as a doctor easier. And finally, these recommendations were never in any way proven to be successful. I am the living proof of that. PS Professor Marshal F. Folstein, Tufts University School of Medicine, Boston, MA, USA is still working and publishing [3] Reference List 1. Patsopoulos NA, Ioannidis JP, Analatos AA. Origin and funding of the most frequently cited papers in medicine: database analysis. BMJ 2006;332:1061-4. 2. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J.Psychiatr.Res. 1975;12:189-98. 3. Qiu WQ, Folstein MF. Insulin, insulin-degrading enzyme and amyloid-beta peptide in Alzheimer's disease: review and hypothesis. Neurobiol.Aging 2006;27:190-8. Competing interests: None declared
On citing scientific papers properly 13 November 2006
Reinhard Wentz, Retired TW2 7PS Send response to journal: Re: On citing scientific papers properly	Madam, In his delightful comments(1) on an earlier paper in the BMJ on citation analysis (2) Dr Nilsson draws attention to the (probably) most frequently cited paper in recent medical history(3). His suggestion for young scientist who think of writing a successful paper (Give it an appealing name, with implication of shortness, quickness, or simply that it will make your life as a doctor easier) could perhaps be paraphrased succinctly: Write a method paper! Method papers are routinely cited more frequently than theoretical or empirical studies, in medical science and other subjects, a phenomenon previously described (4). Dr Nilsson relied on the excellent Web of Knowledge database (WoK) (Institute for Scientific Information (ISI)) to calculate that the paper by Folstein et al. (3) received some amazing 19,250 citations since 1977. I would like to point out that this figure should be increased by at least 500, this being the number of papers citing Folstein incorrectly. Among incorrect references are 300+ from papers which get the page or volume wrong (usually but not exclusively ‘198’ or ‘21’ respectively), many which give an incorrect source journal (incl. suggestions that the paper was published in JAMA) or identify Folstein’s paper as having been published as late as 1997. The WoK has a powerful ‘analyze’ facility which allows to identify institutions which let, in this case, incorrect references slip through their editing process. Each of these institutions has a least ten papers with an incorrect reference to Folstein’s paper: Univ Calif Los Angeles; Harvard Univ; Johns Hopkins Univ; Univ Cambridge; Univ Pittsburgh; Australian Natl Univ; Univ Texas; Univ Washington; Univ Toronto; Columbia Univ. Among periodicals which have a least 10 papers with an incorrect citation are: Neurology; Journal of the American Geriatrics Society; International Journal of Geriatric Psychiatry; Dementia and Geriatric Cognitive Disorders; Journal of Clinical and Experimental Neuropsychology; Archives of Neurology; Aging & Mental Health; Aging Neuropsychology and Cognition; European Journal of Neurology. A further analysis identifying authors who cite the Folstein incorrectly and later authors who then just copied the faulty reference without going to the original publication (or at least checking the reference on PubMed) would be possible. Sadly, Nilsson’s reference will not add to the citation count for Folstein’s paper, as references in BMJ’s Rapid Responses are not visible on the Web of Knowledge. A variant of the FUTON Bias(5), perhaps? Respectfully yours, Reinhard Wentz (1) http://www.bmj.com/cgi/eletters/332/7549/1061#149065 (2) Patsopoulos NA, Ioannidis JP, Analatos AA. Origin and funding of the most frequently cited papers in medicine: database analysis. BMJ 2006;332:1061-4. (3) Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J.Psychiatr.Res. 1975;12:189-98. (4) Peritz BC.Are methodological papers more cited than theoretical or empirical ones? The case of sociology. Scientometrics 1983; 5 :211-18 (5) http://en.wikipedia.org/wiki/FUTON_bias Competing interests: None declared