Review ArticleCombining follow-up and change data is valid in meta-analyses of continuous outcomes: a meta-epidemiological study
Introduction
Many trialists use continuous variables, such as pain intensity or depression severity scores, as clinical outcomes. Variables can be assessed at baseline and follow-up, and estimated treatment effects can be derived either from between-group differences in changes from baseline to follow-up or from a simple comparison of values at follow-up. Point estimates of treatment effects derived from these two approaches are identical if mean baseline values of continuous variables are the same, but will differ if there are baseline imbalances [1]. Standard deviations as measures of distribution of scores will be generally similar if the average correlation between baseline and follow-up values is approximately 0.5. If the correlation is higher than 0.5, then the standard deviations of change data will be smaller; if the correlation is lower than 0.5, then the use of change data will add variation and their standard deviation will be larger than the standard deviation of follow-up data [1]. Differences in point estimates and differences in standard deviations will both affect the estimated standardized mean difference (SMD), expressing differences in point estimates in units of the pooled standard deviation. Significance levels are derived from t-values, which in turn are calculated from the observed difference in point estimates divided by its standard error. Therefore, they will again be influenced by both baseline imbalances and correlation of baseline and follow-up data.
Meta-analysts can use either type of data (follow-up or change) to derive treatment effects from trials included in pooled analyses. In a recent analysis of 10 protocols randomly selected from the Cochrane Library, however, only four protocols specified which one would be used for the calculation of treatment effects [2]. The Cochrane Handbook does not specify which method is preferable, presenting the possibility of data-driven choice of type of data for extraction source. Furthermore, the Cochrane Handbook currently advises against combining the two types of data in a single meta-analysis. If some studies provide only follow-up data and others only change data, the Cochrane Handbook guidance prevents use of all the data. This potentially compromises statistical precision of pooled estimates and prevents the examination of potential sources of heterogeneity in the complete set of trials. If, however, SMD estimates are on average similar in follow-up and change data, power in the analysis will be gained, without introducing bias, by use of all the data.
In addition, the decision to analyze either follow-up or change data may be post hoc and data driven, but the magnitude of bias introduced in meta-analyses by the systematic extraction of data that indicates the largest treatment effect (optimistic selection of SMDs) or, conversely, the data that indicates the smallest treatment effect in each trial (pessimistic selection of SMDs) is unclear. Using data from a meta-epidemiological study of osteoarthritis trials [3], [4], [5], we therefore compared SMDs, mean differences, standard deviations, and significance levels of treatment effects of meta-analyses and their component trials derived from follow-up and change data and determined the extent of bias introduced by optimistic or pessimistic post hoc selection of either type of data in meta-analyses.
Section snippets
Selection of meta-analyses and component trials
Details of the methods used in this meta-epidemiological study are reported elsewhere [4]. We searched The Cochrane Library, Medline, Embase, and CINAHL using database-specific search strategies [4]. We included meta-analyses of randomized or quasi-randomized trials in patients with osteoarthritis of the knee or hip, which assessed patient-reported pain comparing any intervention with placebo, sham, or a nonintervention control. Reports of all component trials were obtained, without language
Characteristics of the included studies
Previous reports describe the study sample and its origin [3], [4]. A total of 21 meta-analyses with 189 trials with 292 randomized comparisons in 41,256 patients were eligible. Table 1 describes the characteristics of the meta-analyses. The median number of trials per meta-analysis was seven (range: 2–29), and the median number of patients per meta-analysis was 1,430 (range: 172–14,579). The pooled treatment effect calculated from follow-up data ranged from −0.05 to −1.37 and the between-trial
Discussion
In our meta-epidemiological study of osteoarthritis trials with pain scores as clinical outcomes, we found no evidence for systematic differences between estimates derived from follow-up and change data. Differences in SMDs between follow-up and change data, mean differences and standard deviations measured on original scales, and corresponding P-values were all scattered around the null. This suggests that there are no a priori reasons that prevent the pooling of SMDs derived from follow-up
References (17)
- et al.
Osteoarthritis: rational approach to treating the individual
Best Pract Res Clin Rheumatol
(2006) - et al.
Meta-analysis in clinical trials
Control Clin Trials
(1986) Issues in the use of change scores in randomized trials
J Clin Epidemiol
(1989)- et al.
Statistics notes: analysing controlled trials with baseline and follow up measurements
BMJ
(2001) - et al.
Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study
BMJ
(2009) - et al.
The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study
Arthritis Rheum
(2009) - et al.
The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study
BMJ
(2009) - et al.
Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study
BMJ
(2010)
Cited by (0)
Funding: This project is supported by a grant from the ARCO Foundation, Switzerland.
Conflict of Interest: None.