Intended for healthcare professionals


Will the ROB-ME checklist prevent omission bias in meta-analyses?

BMJ 2023; 383 doi: (Published 20 November 2023) Cite this as: BMJ 2023;383:p2653

Linked research methods & reporting

ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis

  1. Nick Freemantle, professor of clinical epidemiology and biostatistics1,
  2. Alar Irs, chief medical officer2
  1. 1Comprehensive Clinical Trials Unit, Institute of Clinical Trials and Methodology, University College London, London WC1V 6LJ, UK
  2. 2Estonian State Agency of Medicines (Ravimiamet), Tartu, Estonia
  1. Correspondence to: N Freemantle nicholas.freemantle{at}

Tool’s objectives are clear and important, but validation might be difficult

The checklist industry has produced another output, the ROB-ME instrument for assessing risk of bias due to missing evidence in pairwise meta-analyses, nestling between ROB-MEN for network meta-analyses and RoB 2 for assessing bias in the reporting of trials (doi:10.1136/bmj-2023-076754).1 Selective reporting of study results is a well known source of bias in meta-analyses, and ROB-ME is the first structured approach for assessing the risk of bias that arises when entire studies, or particular results within studies, are missing from a meta-analysis because of the P value, magnitude, or direction of the study results.

The tool intends to help researchers select and define the meta-analyses to be assessed, identify the trials that might have missing results, and crucially consider the potential for missing studies across the review. In ideal circumstances when a meta-analysis includes all trials that have been conducted on a question, the results are an unbiased best estimate of treatment effect. Well conducted trials on the same topic differ on the basis of the play of chance (the mechanism by which treatment was allocated between participants), and it is obvious that if we select only those trials with results that (by chance) are more …

View Full Text

Log in

Log in through your institution


* For online subscription