Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sourcesBMJ 2005; 331 doi: https://doi.org/10.1136/bmj.38636.593461.68 (Published 03 November 2005) Cite this as: BMJ 2005;331:1064
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Greenhalgh et al (1) describe their experience reviewing published
evidence for health care topics other than straightforward questions of
clinical efficacy and safety. While we concur that such topics pose a
greater challenge to researchers' ability to anticipate exact search and
selection criteria prospectively, we believe that approaches can be
developed which remain systematic rather than opportunistic. Manual checks
of references in accepted papers are, we agree, a key source of important
data – they can also serve as a test of the original electronic search
We suggest and have used an iterative approach (2) whereby all
manually or "serendipitously" found relevant articles are checked to
identify the reasons they were not captured in the initial search
(publication in a journal not indexed in the electronic databases
searched, not indexed under search terms used, etc.). A systematic remedy
can then be put into place, via protocol amendment, ensuring that
similarly relevant papers (perhaps with different findings) also have the
highest chance of being found. This allows confidence that appropriate
efforts to reduce bias have been made in the review.
Incidentally, the good news for researchers is that papers not found
due to complete lack of electronic indexing are increasingly uncommon.
More typically in our experience, search terms need to be broadened to
become more comprehensive – resulting in large numbers of abstracts to
weed through, but maintaining the systematic nature of the research.
1. Greenhalgh T, Peacock R. Effectiveness and efficiency of search
methods in systematic reviews of complex evidence: audit of primary
sources. BMJ 2005 Nov 5;331(7524):1064-5.
2. Frame D, Klawansky S. Increasing PubMed yield for pharmacoeconomic
research using iterative search strategies. ISPOR Connections 2003; 9
Competing interests: No competing interests