Original Article
Imputing missing standard deviations in meta-analyses can provide accurate results

https://doi.org/10.1016/j.jclinepi.2005.06.006Get rights and content

Abstract

Background and Objectives

Many reports of randomized controlled trials (RCTs) fail to provide standard deviations (SDs) of their continuous outcome measures. Some meta-analysts substitute them by those reported in other studies, either from another meta-analysis or from other studies in the same meta-analysis. But the validity of such practices has never been empirically examined.

Methods

We compared the actual standardized mean difference (SMD) of individual RCTs and the meta-analytically pooled SMD of all RCTs against those based on the above-mentioned two imputation methods in two meta-analyses of antidepressants.

Results

Two meta-analyses included 39 RCTs of fluoxetine (n = 3,681) and 25 RCTs of amitriptyline (n = 1,832), which had actually reported means and SDs of the Hamilton Rating Scale for Depression. According to either of the two proposed imputation methods, the agreement between actual SMDs and imputed SMDs for individual RCTs was very good with ANOVA intraclass correlation coefficients between 0.61 and 0.97. The agreement between the actual pooled SMD and the imputed one was even better, with minimal differences in both their point estimates and 95% confidence intervals.

Conclusion

For a systematic review where some of the identified trials do not report SDs, it appears safe to borrow SDs from other studies.

Introduction

Conduct of a systematic review or a meta-analysis involves comprehensive search of relevant randomized controlled trials (RCTs) and their quantitative or qualitative synthesis. To pool results on a continuous outcome measure of the identified RCTs quantitatively, one needs both means and standard deviations (SDs) on that outcome measure for each RCT.

Many reports of RCTs, however, fail to provide SDs for their continuous outcomes. It is sometimes possible to use P or t or F values, reported in the original RCTs, to calculate exact SDs [1]. When none of these is available, it is recommended that one should contact primary authors [2]. However, the yield is very often very low; some are incontactable, some never respond, and others report that the data are discarded, lost or irretrievable because there are no longer any computers to read the tapes.

Some meta-analysts then resort to substitution of SDs of known outcome measures by those reported in other studies, either from another meta-analysis or from other studies in the same meta-analysis. But the validity of such practices has never been empirically examined.

The present article therefore aims to examine empirically the validity of borrowing SDs from other studies when individual RCTs fail to report SDs in a meta-analysis, by simulating the above-mentioned two imputation methods for SDs in two meta-analyses on antidepressants that we have conducted [3], [4]. Systematic reviews for depression are particularly suitable for this purpose, because Hamilton Rating Scale for Depression [5] (HRSD) is the de facto standard in symptom assessment and is used in many depression trials identified for overviews.

Section snippets

Imputation from a previous meta-analysis

We used the pooled SD for the 17-item HRSD and 21-item HRSD from a comprehensive meta-analysis of all three-armed trials for depression involving an investigational drug, an active comparison drug, and placebo [6] for two reasons. We are unaware of any other meta-analysis that dealt with the whole range of antidepressants along with placebo for depression. We argue that such a meta-analysis would provide a more impartial estimate of SDs for ratings scales than, for example, a meta-analysis

Characteristics of individual RCTs in the two meta-analyses

Of 133 RCTs pooled to examine the overall efficacy for fluoxetine vs. other antidepressants, 108 used HRSD, 8 used Montgomery-Asberg Depression Rating Scale (MADRS) [13], and 14 used other scales. Of the first group, 36 used 17-item HRSD, 39 used 21-item HRSD, and the remaining 33 used other or unknown versions of HRSD. Moreover, of the 36 studies that used HRSD-17, only 24 reported SDs; of the 39 which used HRSD-21, only 15 reported SDs. We therefore compared the meta-analytic results for 39

Discussion

Few studies to date have dealt with the problem of missing variance estimates for continuous variables in meta-analyses, although apparently the problem is annoyingly common. For example, in a systematic review of sodium reduction on blood pressure, fewer than half of the identified trials (10 out of 26 identified trials) published a variance estimate or information to allow one to derive it [14]. In the comprehensive meta-analysis of three-armed studies involving antidepressants [6], the

References (18)

There are more references available in the full text version of this article.

Cited by (1083)

View all citing articles on Scopus
View full text