Intended for healthcare professionals

CCBYNC Open access

Rapid response to:

Research

Prophylactic antibiotics for burns patients: systematic review and meta-analysis

BMJ 2010; 340 doi: https://doi.org/10.1136/bmj.c241 (Published 15 February 2010) Cite this as: BMJ 2010;340:c241

Rapid Response:

Should we be wary of statistics in medicine?

I read with great interest and excitement the systematic review and
meta-analysis on prophylactic antibiotics for burns patients.

Looking at it closer, a long-standing question seems to emerge and it
is that of validity and generalisability, which is often encountered with
even well-conducted systematic reviews and meta-analyses

The inclusion of quasi-randomised controlled trials in this meta-
analysis does probably not remove anything from its value and a trade-off
between the trueness and the validity of findings is undoubtedly often
needed before conclusions are made.

Heterogeneity is probably not avoidable to a certain extent as
systematic reviews bring together studies that are diverse both clinically
and methodologically (1).

I still wonder if the question of heterogeneity is not inadequately
addressed here.

The Cochran’s Q value gives you only a dichotomous answer
(heterogeneous or not) but the I2 quantifies the effect of heterogeneity,
providing a measure of the degree of inconsistency in the studies’ results
(2).

Does the I2 value of 32% reflect a low, moderate or high
heterogeneity then? And how much is too much heterogeneity?

There is no hard rule to define the above but one generally accepted
rule of thumb characterizes an I2 of less than 25% as small heterogeneity,
25 to 50% as moderate, and more than 50% as large heterogeneity.

When the authors conclude that systemic prophylaxis is associated
with a significant reduction in all cause mortality without significant
heterogeneity, one could argue that the studies included are not
consistent enough to draw hard clinical conclusions.

The authors do recognise in their discussion that all types and
degree of burns were included and a subgroup analysis should have been
carried but unfortunately this was impossible because of paucity of
trials.

I wonder how generalisable their findings are on the general
population with the included studies being moderately heterogeneous.

References :

(1) Higgins J, Thompson S, Deeks J, Altman D. Statistical
heterogeneity in
systematic reviews of clinical trials: a critical appraisal of guidelines
and
practice. J Health Serv Res Policy 2002;7:51-61.

(2) 4 Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-
analysis. Stat Med 2002;21:1539-58.

Competing interests:
None declared

Competing interests: No competing interests

19 March 2010
Janos P Baombe
emergency doctor
Emergency Department, Manchester Royal Infirmary, Oxford Road, Manchester M13 9WL