Inappropriate referencing in research

BMJ 2009; 339 doi: http://dx.doi.org/10.1136/bmj.b2049 (Published 21 July 2009) Cite this as: BMJ 2009;339:b2049
  1. Dean Fergusson, senior scientist
  1. 1Clinical Epidemiology Program, General Campus, Ottawa Health Research Institute, Box 201, 501 Smyth Rd, Ottawa, ON K1H 8L6, Canada
  1. dafergusson{at}ohri.ca

    Has serious consequences, and the research community needs to act

    During the preparation and writing of manuscripts, protocols, grant submissions, technical reports, and conference abstracts, authors must consider carefully the selection, completeness, and appropriateness of the articles referenced. Improper citation is not a benign practice; adequate and accurate citation is a necessity of scientifically and methodologically sound research. Rather than treating citation errors in a particular journal article as isolated incidents, we must appreciate that such errors can be replicated in further articles and, therefore, cause considerable damage over time. Incorrect information can be promoted, alternative evidence ignored, and redundant research undertaken following inappropriate use of references, impairing scientific progress and affecting patient care.

    In the linked study (doi:10.1136/bmj.b2680), Greenberg illustrates a number of serious consequences of inappropriate or inaccurate citation of published scientific work. Greenberg tracks the citation history of the hypothesis that amyloid β is “produced by and injures skeletal muscle fibres of patients with inclusion body myositis”.1 He concludes that the publication and respective citation history for this hypothesis offers empirical evidence that inappropriate or inaccurate citation can cause serious distortions, including bias, amplification, and invention. Erroneous and unfounded claims can be perpetuated, which sets back real scientific progress and has direct implications on how patients are treated.

    A clear message from Greenberg’s work is that investigators have an obligation to critically appraise existing evidence and develop their own interpretation of the results of individual studies. Although explanations provided in published studies are helpful and insightful, we must not rely solely on the word of the authors. In articles referencing a previous study, the interpretation of this work provided in the text may be insufficient. Although tempting, we must not rely on descriptions provided by authors citing a primary paper, who may lack or be ignorant of primary results. Ideally, the data that form the basis of claims in a paper should be replicated from the primary source, either in the article or an appendix. As imparted by teachers and mentors in research, we as authors must always address the scientific and methodological limitations of our findings. Incorporation of these limitations should accompany any future citation, as their absence can bias interpretation.

    Greenberg also shows the fundamental need for systematic reviews in all types of research. By selectively citing studies or unsystematic reviews that suit a particular hypothesis, a bias so serious that the recommendations put forth are erroneous can be introduced. We can no longer continue to choose and selectively report work that does not represent the spectrum of available evidence.

    Traditionally, the majority of systematic reviews have concentrated on summarising clinical evidence of effectiveness or efficacy provided by randomised clinical trials conducted in humans. The utility of systematic reviews extends well beyond questions of clinical effectiveness. Through a systematic and transparent review process—including literature searches, eligibility criteria, and quality appraisal—they provide a less biased presentation and summary of the evidence base than traditional narrative reviews.

    Indeed, basic research would also benefit from systematic reviews to justify and support proposed hypotheses and respective methods. Problems such as inadequate sample size, poor study design, lack of blinding, and lack of randomisation hold true for animal research as well as for clinical research. Although the methods and protocols for conducting systematic reviews in basic research lag far behind those for clinical research in humans, these limitations do not preclude their conduct.

    Some may argue that Greenberg’s research assesses but a single example of improper citation. Further research investigating the extent and impact of citation bias is warranted and extensive preventive measures to ensure proper procedures are followed should be introduced. We do not require a body of evidence that demonstrates wrong doing; instead, we need assurances that biases are minimised in the preparation of scientific documents.

    Firstly, for any grant proposal, investigators must incorporate a systematic review to justify their proposed hypothesis and objectives. Investigators can either integrate a published systematic review or, if absent, undertake their own. Doing so at least ensures that the proposed hypothesis is relevant; the methodological and scientific limitations and strengths of previous research are considered; all previous evidence is weighed and incorporated; and a transparent path of literature review is provided to peer reviewers and non-peer reviewers.

    Certain grant organisations strongly encourage a systematic review for all clinical trial proposals.2 3 A systematic review should, however, be required for any research protocol, regardless of hypothesis, study design, or types of participants. This measure requires a large and, for some, uncomfortable step forward, but it is a necessary step. Indeed, the need for systematic reviews extends beyond the issue of citation bias.

    Secondly, journals should require corresponding authors to formally acknowledge that they take responsibility for the appropriateness and accuracy of their manuscript’s reference list. This measure should raise awareness of the seriousness of improper citation. The acknowledgment would be comparable to the current practice at journals for authors to pronounce their roles in data acquisition, statistical analysis, drafting of the manuscript, and study interpretation. Compared with the current situation in which citation review is left only to the editorial office, a simple check box could ensure that authors are accountable for the completeness, accuracy, and interpretation of references. In turn, this measure could produce stronger manuscripts that are less prone to citation bias and its deleterious downstream effects.

    Research hypotheses require robust evidence along with a strong rationale, and health professionals require a complete and balanced account of the evidence base to better guide patient care. Scientific progress is set back by faulty hypotheses and redundant research that is propagated by selective and erroneous citation practices. The research community must attend to the issue of citation bias with some urgency.


    Cite this as: BMJ 2009;338:b2049