Education and debate
Matthias Egger, Jonathan AC Sterne, George Davey Smith
Over the past few years, computer software entirely devoted to meta-analysis has increasingly become available, and meta-analytic procedures have been introduced in general statistical software packages. We have identified five commercial packages which are sold with handbooks and three programs which are in the public domain. In this article we briefly review each program and, in table 1 and table 2, provide a comparative summary of features.
| Table 1 Summary of features of five commercial meta-analysis software packages | |||||
| | FAST*PRO | STATA | True Epistat | DSTAT | DESCARTES |
| Version tested | 1.0 | 5.0 | 5.0 | 1.10 | available in spring 1998 |
| Operating system | MS-DOS | Windows | MS-DOS | MS-DOS | Windows |
| Distributor | Academic Press 24-28 Oval Road London NW1 7DX UK 1 |
Timberlake Consultants 47 Hartfield Crescent West Wickham Kent BR4 9DW UK 2 www.stata.com |
Epistat Services 2011 Cap Rock Circle Richardson Texas 75080-3417 USA | Lawrence Erlbaum
Ass. 365 Broadway Hillsdale New Jersey 07642 USA | Update Software Ltd Summertown Pavilion Middle Way Summertown Oxford OX2 7LG UK www.update-software.com |
| Price | £ 250 | £ 345 | US$ 533 | US$ 99 | not yet available |
| Input data | 2 x k tables, odds ratios, relative risks, risk difference, differences of means, slopes and others | Any (package has sophisticated data management and modelling facilities). Data are analysed using standard methods and the results used as input for meta analysis commands. | 2 x 2 tables, differences of means, p values, z-, t-, F-, 02- statistics, correlation coefficients | 2 x 2 tables, means, p values, z-, t-, F-, 02- statistics, correlation coefficients | 2x2 tables, means, covariates |
| Statistical models | Bayesian and classical fixed effects and random effects | Fixed effects and random effects. Empirical Bayes. Regression models. | Fixed effects and random effects | Fixed effects only | Fixed and random effects. Regression models. |
| Output effect measures | Odds ratio, relative risk, risk difference, difference of means and others | Odds ratio, relative risk, risk difference, difference of means and others | Odds ratio, relative risk, risk difference, standardised difference and others | Standardised difference | Odds ratio, relative risk, risk difference, NNT, weighted mean difference, standardised mean difference |
| Test for Homogeneiy | Yes | Yes | Yes | Yes | Yes |
| Manual | Yes | Yes | Yes | Yes | Yes |
| Formulae given | No | Yes3 | Yes | Yes | Yes |
| Technical support is available from Dr Vic Hasselblad, Durham, NC, USA (Fax +1 919 286 5601) The meta-analysis macro for use within STATA is free and can be downloaded from http://www.stata.com Sharp SJ, Sterne JAC. Sbe16: Meta-analysis. Stata Technical Bulletin 1997;38: 9-14 | |||||
STATA
Stata is a comprehensive statistics, data management and graphics package for which a meta-
analysis command has recently been written. (9) Individual-level or study-level data are analysed
using standard methods to provide an effect estimate (e.g. odds ratio, risk difference or
difference between means) and corresponding standard error for each study. The meta-analysis
command then calculates fixed-effects (variance-weighted) and random-effects (DerSimonian-
Laird (6)) estimates, together with the standard 02 test for heterogeneity between studies and
estimate of between-study variance. The classical meta-analysis graph is displayed with either
the fixed-effects or random-effects combined estimate. Empirical Bayes estimates of the true
effect in each study given the random-effects model can be calculated, displayed and graphed.
Results and graphical displays can be shown either on the original scale or on the ratio scale
(when the original effect estimates are on a log scale). Funnel plots can be displayed using the
standard graphics facilities of the package. A meta-regression (10,11) command which can be
used to explore sources of heterogeneity between studies has been written and will be available
soon. Cumulative meta-analysis will also be available soon.
True Epistat
This is a statistics package which also offers a number of meta-analysis capabilities. Studies
comparing two groups and using odds ratios, relative risks, risk differences (dichotomous
outcomes), or standardised differences (continuous outcomes) can be analysed in a variance-
weighted fixed effects model or a DerSimonian-Laird random effects model. Data are entered in
a two-by-two table or as group variances along with the difference between two means.
Correlation coefficients, test statistics from widely used distributions and p-values (and
mixtures of the former) can also be combined. Cumulative meta-analysis is not available. The
results are given in tabular form or in the typical graphical display showing effect measures and
confidence intervals for each study and for the overall result. Funnel plots can be drawn. The
graphics can be edited on the screen and printed.
DSTAT
DSTAT was developed for meta-analysis in the psychological sciences. It combines studies
comparing two groups. The data are entered as correlation coefficients, test statistics, p-values
or mixtures. These statistics are then converted into a standardised (scale-free) effect measure,
the effect size (Hedges' g) which is defined as the difference between the two groups expressed
in (pooled) standard deviation units (12). If the user wishes to do so, a bias-adjusted effect size
can be calculated (Hedges' d). Adjusted or unadjusted effect sizes of individual studies are then
combined to produce an overall value. Clinically more relevant quantities such as the difference
in risk, the relative risk or the odds ratio cannot be calculated with DSTAT. Also, results cannot
be graphically displayed. These drawbacks limit the usefulness of DSTAT for meta-analysis in
medical research.
DESCARTES
DESCARTES is a set of software tools for writing systematic reviews and performing meta-
analysis which is being developed by Update Software, the company which is responsible for
the Cochrane Library and the Cochrane Collaboration's RevMan package (see below). The
package will be released in spring 1998. It is an interactive guide through all the steps involved,
from protocol through data collection and analysis to publication in paper and electronic
formats. The emphasis is on producing a finished document (protocol, systematic review or
individual meta-analysis) so output is geared towards publication quality graphics, text and
tables. Data for meta-analyses can be imported from other packages (for example RevMan),
entered directly into a spreadsheet or entered interactively via guided data-entry screens which
check for impossible or unlikely values. Standard fixed effects and random effects models and
meta-regression models are available for dichotomous, continuous and individual patient data.
Graphical output allows a wide range of plots (for example funnel, L'Abbé, (13) and Galbraith (14)
plots). Cumulative meta-analysis and sensitivity analyses are also available. Suggested
interpretations of calculated statistics can be generated automatically in textual form and
included in the output. All DESCARTES output can be pasted directly into other Windows
packages.
| Table 2 Summary of features of three public domain meta-analysis software packages. | |||
| | RevMan | Easy MA | Meta-Analyst |
| Version tested | 3.0 | 97b | 0.988 |
| Operating system | Windows | MS-DOS | MS-DOS |
| Distributor | The Cochrane Collaboration | Dr M Cucherat Dept. of Clinical Pharmacology 162 Av. Lacassagne F-69003 Lyon France | Dr J Lau New England Medical Center, Box 63 750 Washington St Boston, MA 02111 USA |
| Internet address | hiru.mcmaster.ca/cochrane | www.spc.univ-lyon1.fr/citccf/easyma www.spc.univ-lyon1.fr/~mcu/easyma | available from Dr Lau via electronic mail (joseph.lau{at}es.nemc.org) |
| Input data | 2 x 2 tables, means | 2 x 2 tables | 2 x 2 tables |
| Statistical models | Fixed effects and random effects | Fixed effects and random effects | Fixed effects and random effects |
| Output effect measures | Odds ratio, relative risk, risk difference | Odds ratio, relative risk, risk difference, NNT | Odds ratio, relative risk, risk difference |
| Test for Homogenei y | Yes | Yes | Yes |
| Manual | Yes | Yes | No |
| Formulae given | in preparation | Yes | No |
Easy MA
EasyMA was developed by Michel Cucherat from the University of Lyon. It is described in
more detail elsewhere. (16) The package can be down loaded from an internet site (table 2). All
menu headings are written in English but contextual help is at present available only in French.
An English version is in preparation. EasyMA was developed for meta-analysis of clinical trials
with two arms and one or several dichotomous outcomes. It is menu driven and offers fixed
effects (e.g. Yusuf-Peto, (5) Mantel-Haenszel (4)) and random effects models (e.g. DerSimonian
and Laird (6)) for calculation of overall odds ratios, relative risks and risk differences. In the latter
case the number of patients needed to treat to prevent one event (NNT) (1,17) is also given. Other
useful features include a table ranking studies according to control group event rates, and
weighted and unweighted regression analysis of control group against treatment group rates.
EasyMA produces the classical meta-analysis graphs both for standard and cumulative meta-
analysis as well as radial and funnel plots. Rosenthal's number of unpublished negative trials
needed to render the combined results non-significant (18) and Begg and Mazumdar's test for
publication bias (19) are also available. Graphs are of high quality but editing possibilities are
somewhat limited.
Meta-Analyst
Meta-analyst was programmed by Joseph Lau from the New England Medical Center.
Interested readers should write to the author if they would like to obtain a copy (see table for
details). Like EasyMA, this software was developed for conventional and cumulative meta-
analysis of clinical trials with two arms and dichotomous outcomes. Only one outcome can be
entered at a time. The programme is easy to use and offers the widely used fixed effects and
random effects models for combining odds ratios, relative risks and risk differences. The
graphs produced are of excellent quality, but, as in EasyMA, they cannot easily be exported for
editing in another programme. Additional features include regression analyses of treatment
effect against control event rates and sample size estimation of a hypothetical future randomised
trial.
| Table 3 Data used in meta-analysis of 17 trials of the effect of beta-blockade on mortality after myocardial infarction | ||||
| Trial | Beta-blocker | Control | ||
| | N | No of deaths | N | No of deaths |
| Reynolds | 38 | 3 | 39 | 3 |
| Wilhemsson | 114 | 7 | 116 | 14 |
| Ahlmark | 69 | 5 | 93 | 11 |
| MIS | 1533 | 102 | 1520 | 127 |
| Baber | 355 | 28 | 365 | 27 |
| Ahnve | 59 | 4 | 52 | 6 |
| Norwegian MC | 945 | 98 | 939 | 152 |
| Taylor | 632 | 60 | 471 | 48 |
| Hansteen | 278 | 25 | 282 | 37 |
| BHAT | 1916 | 138 | 1921 | 188 |
| Julian | 873 | 64 | 583 | 52 |
| Austr & Swed | 263 | 45 | 266 | 47 |
| Manger Cats | 291 | 9 | 293 | 16 |
| EIS | 858 | 57 | 883 | 45 |
| Olsson | 154 | 25 | 147 | 31 |
| LIT | 1195 | 65 | 1200 | 62 |
| Boissel | 298 | 17 | 309 | 34 |
Department of Social Medicine
University of Bristol
Bristol BS8 2PR,
U.K.
Matthias Egger,
Reader in Social Medicine and Epidemiology
George Davey Smith,
Professor of Clinical Epidemiology
Department of Public Health Medicine
United Medical and Dental Schools
St Thomas's Hospital
London
Jonathan AC Sterne,
Senior Lecturer in Medical Statistics
Address for correspondence:
Dr Matthias Egger
Department of Social Medicine
University of Bristol
Canynge Hall,
Whiteladies Road
Bristol BS8 2PR,
UK
Tel 0117 928 73 87
Fax 0117 928 73 25
Email m.egger@bristol.ac.uk
References
1. Egger M, Davey Smith G, Phillips AN. Meta-analysis: principles and procedures. BMJ
1997; 315:1533-1537. [Full text]
2. Eddy DM, Hasselblad V, Shachter R. Meta-analysis by the confidence profile method.
The statistical synthesis of evidence. Boston: Academic Press, 1992.
3. Lilford RJ, Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue.
BMJ 1996; 313:603-607.
4. Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies
of disease. J Natl Cancer Inst 1959; 22:719-748.
5. Yusuf S, Peto R, Lewis J, Collins R, Sleight P. Beta blockade during and after myocardial
infarction: an overview of the randomized trials. Progr Cardiovasc Dis 1985; 17:335-371.
6. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials 1986;
7:177-188.
7. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative
meta-analysis of therapeutic trials for myocardial infarction. New Engl J Med 1992; 327:248-
254.
8. Egger M, Davey Smith G. Meta-analysis: potentials and promise. BMJ 1997; 315:1371-
1374.[Full text]
9. Sharp SJ, Sterne JAC. sbe16: Meta-analysis. Stata Technical Bulletin 1997; 38:9-14.
10. Thompson S.G. and Sharp S.J. 1997. Explaining heterogeneity in meta-analysis: a
comparison of methods. Stat Med (submitted).
11. Egger M, Davey Smith G, Schneider M, Minder CE. Bias in meta-analysis detected by a
simple, graphical test. BMJ 1997; 315:629-634. [Full text]
12. Hedges LV, Olkin I. Statistical methods for meta-analysis. Boston: Academic Press,
1985.
13. L'Abbe KA, Detsky AS, O'Rourke K. Meta-analysis in clinical research. Ann Intern Med
1987; 107:224-233.
14. Galbraith R. A note on graphical presentation of estimated odds ratios from several clinical
trials. Stat Med 1988; 7:889-894.
15. Oxman A. VI. Preparing and maintaining systematic reviews. In: Sackett DL, Oxman
A, editors. Cochrane Collaboration Handbook. Oxford: The Cochrane Collaboration, 1995:
16. Cucherat M, Boissel J-P, Leizorovicz A, Haugh MC. EasyMA: a program for the meta-
analysis of clinical trials. Computer Methods and Programs in Biomedicine 1997; 53:187-
190.
17. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the
consequences of treatment. New Engl J Med 1988; 318:1728-1733.
18. Rosenthal R. The 'file drawer problem' and tolerance for null results. Psychol Bull 1979;
86:638-641.
19. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication
bias. Biometrics 1994; 50:1088-1099.
20. Egger M, Davey Smith G. Meta-analysis: bias in location and selection of studies. BMJ
1998; 316:61-66.