Letters The automation of systematic reviews

The efficient production of high quality evidence reviews is important for the public good

BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f846 (Published 13 February 2013) Cite this as: BMJ 2013;346:f846
  1. Julian H Elliott, head, clinical research1,
  2. Chris Mavergames, director of web development2,
  3. Lorne Becker, emeritus professor3,
  4. Joerg Meerpohl, deputy director2,
  5. Jessica Thomas, IMS team manager4,
  6. Russell Gruen, director5,
  7. David Tovey, editor in chief6
  1. 1Infectious Diseases Unit, Alfred Hospital, Melbourne, Vic, Australia
  2. 2German Cochrane Centre, Freiburg, Germany
  3. 3Department of Family Medicine, SUNY Upstate Medical University, Syracuse, NY 13210, USA
  4. 4Cochrane IMS Team, Nordic Cochrane Centre, Copenhagen, Denmark
  5. 5National Trauma Research Institute, Alfred Hospital, Melbourne, Vic, Australia
  6. 6Cochrane Editorial Unit, Cochrane Collaboration, London, UK
  1. julian.elliott{at}alfred.org.au

We support Tsafnat and colleagues’ argument that automation could transform systematic review production.1 Technological innovation has played a central role since the birth of systematic review,2 and machine processes are embedded throughout current systematic review workflow. Nevertheless, current methods are not sustainable in the face of the deluge of primary research.

Few of the recent innovations that the authors highlight have been applied in the real world. The development of viable tools has been hampered by performance that is poorer than that demanded by users with rigorous methodological standards. We therefore need vibrant environments for systematic review innovation and incentive structures that support rapid and broad release.

As long as systematic review relies on the analysis of unstructured data and text, human input will be crucial. We should therefore focus not just on “making the machines work harder,” but on creating the best partnership between people and machines.3 The Cochrane Collaboration is working to optimise human effort by using the PICO structure to link reports, studies, reviews, and external data sources in data repositories.4 The relationship between the new Cochrane register of studies and Cochrane’s review writing software, RevMan, is advancing towards semi-automated inclusion into reviews of extracted data stored in a common data repository.

The efficient production of high quality evidence reviews is important for the public good. With ongoing and diverse innovations, we believe that the trade offs between methodological rigour and review currency can be eroded, to result in “living systematic reviews”—high quality online evidence summaries that are dynamically updated as new evidence becomes available.5


Cite this as: BMJ 2013;346:f846


  • Competing interests: All authors are employed by or are active contributors to the Cochrane Collaboration.


View Abstract

Log in

Log in through your institution


* For online subscription