Variability among studies should be investigated

BMJ 1995;310:1604 (17 June)

Letters

Variability among studies should be investigated

EDITOR,--Matthias Egger and George Davey Smith explain how poolingof smaller studies can turn out to be misleading after the subsequentpublication of larger trials and suggest that meta-analysesof small trials should now be distrusted, even if their combinedeffect reaches significance.¹ Yet they also point out that evidencefrom megatrials will continue to be unavailable for most medicalinterventions, and hence meta-analyses of smaller trials remainthe only apparent way of appraising available evidence.

The various statistical manoeuvres that the authors suggestto remedy this situation ignore one important focus of meta-analysis--namely,the degree of variability, or spread, among different studies.One way of analysing this is to perform a {chi} ² test of deviationsof individual studies from the overall mean achieved by poolingthe studies.² A significant P value for this test suggests aconsiderable amount of variability around the summary result.This approach could provide a more . . . [Full text of this article]