Rapid Responses to:

INFORMATION IN PRACTICE: Trisha Greenhalgh and Richard Peacock Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources BMJ 2005; 331: 1064-1065 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Systematic review methods: Becoming less dependent on "the coffee queue"

Diana Frame, Susan D. Ross, MD, FRCPC   (29 November 2005)

Systematic review methods: Becoming less dependent on "the coffee queue" 29 November 2005
Diana Frame, Director, Information Services MetaWorks Inc., Medford MA 02155, Susan D. Ross, MD, FRCPC Send response to journal: Re: Systematic review methods: Becoming less dependent on "the coffee queue"	Greenhalgh et al (1) describe their experience reviewing published evidence for health care topics other than straightforward questions of clinical efficacy and safety. While we concur that such topics pose a greater challenge to researchers' ability to anticipate exact search and selection criteria prospectively, we believe that approaches can be developed which remain systematic rather than opportunistic. Manual checks of references in accepted papers are, we agree, a key source of important data – they can also serve as a test of the original electronic search strategy. We suggest and have used an iterative approach (2) whereby all manually or "serendipitously" found relevant articles are checked to identify the reasons they were not captured in the initial search (publication in a journal not indexed in the electronic databases searched, not indexed under search terms used, etc.). A systematic remedy can then be put into place, via protocol amendment, ensuring that similarly relevant papers (perhaps with different findings) also have the highest chance of being found. This allows confidence that appropriate efforts to reduce bias have been made in the review. Incidentally, the good news for researchers is that papers not found due to complete lack of electronic indexing are increasingly uncommon. More typically in our experience, search terms need to be broadened to become more comprehensive – resulting in large numbers of abstracts to weed through, but maintaining the systematic nature of the research. 1. Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 2005 Nov 5;331(7524):1064-5. 2. Frame D, Klawansky S. Increasing PubMed yield for pharmacoeconomic research using iterative search strategies. ISPOR Connections 2003; 9 (5):4-5. Competing interests: None declared