Statistics notes

Interaction 3: How to examine heterogeneity

^a Department of Medical Statistics, University of Newcastle, Newcastle upon Tyne NE2 4HH, ^b ICRF Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, PO Box 777, Oxford OX3 7LF

Correspondence to: Dr Matthews.

In preceding Statistics Notes we introduced the concept of interaction¹ and explained why a common approach to the assessment of interaction is incorrect.² In this note we give details of the correct approach using the same two examples.

In a study of the effect of maternal vitamin D supplementation on neonatal serum calcium concentrations³ the researchers were interested in the possible difference between the effect of supplementation on breast and bottle fed babies. We define the treatment effect in each feeding group to be the difference in the mean serum calcium concentration of babies receiving supplements and those receiving placebo in that group: the treatment means and observed effects in the feeding groups are given in table 1.

Table 1--Serum calcium concentrations (mmol/l) at 1 week in babies born to mothers
given vitamin D supplements or placebo and analysed according to whether they were
breast fed or bottle fed
--------------------------------------------------------------------------------------
                                   Breast fed                     Bottle fed
--------------------------------------------------------------------------------------
Serum calcium             Supplement         Placebo     Supplement        Placebo
--------------------------------------------------------------------------------------
Treatment mean               2.45             2.41         2.30             2.20
Standard error               0.036            0.032        0.022            0.019
No                            64               102          169              285
Treatment effect                     0.04                          0.10
Standard error                       0.048                         0.029
P value                              0.40                          0.0006

The first step is to compute the difference between the two treatment effects--that is, 0.10 - 0.04 = 0.06 mmol/l. The standard error of this difference is 0.056 mmol/l, found from the standard errors of the separate effects using the usual method for the standard error of a difference.⁴ This is the same method that provides the standard error of a treatment effect from the standard errors of the treatment means. The P value can found from the ratio of the difference to its standard error, namely 0.06/0.056 = 1.07, again using standard methods,⁴ which gives P = 0.28, showing there is no evidence that the effects are different between the two feeding groups. An approximate 95% confidence interval can be found for the difference in the treatment effects in the usual way,⁴--that is, as 0.06 +/- 1.96 x 0.056, or - 0.05 to 0.17 mmol/l.

A similar approach is adopted with a binary outcome measure. In a controlled trial of antenatal steroid therapy for neonatal respiratory distress syndrome 27.3% (9/33) of babies born to mothers with pre-eclampsia and 14.1% (37/262) of babies born to mothers without pre-eclampsia in the control group developed neonatal respiratory distress syndrome; the corresponding figures in the steroid group were 21.2% (7/33) and 7.9% (21/267) respectively.⁵ Once standard errors of each of these percentages have been found in the usual way⁴ the method for assessing an interaction between steroid therapy and mother's pre-eclampsia is the same as for continuous outcomes. The treatment effect in babies of mothers with pre-eclampsia is 27.3 - 21.2 = 6.1% (standard error 10.5%) and in babies born to unaffected mothers it is 14.1 - 7.9 = 6.2% (standard error 2.7%), so the difference in treatment effects is 6.2 - 6.1 = 0.1% (standard error 10.9%), from which the P value for the difference in treatment effects is P = 0.99. Thus there is no evidence in this trial that the effect of antenatal steroids depends on whether the mother suffered from pre-eclampsia: the 95% confidence interval for the difference in the treatment effects can also be constructed as before, giving 0.1 +/- 1.96 x 10.9 or - 21.3% to 21.5%.

1. Altman DG, Matthews JNS. Interaction 1: heterogeneity of effects. BMJ 1996;313:486. [Free Full Text]
2. Matthews JNS, Altman DG. Interaction 2: compare effect sizes not P values. BMJ 1996;313:808. [Free Full Text]
3. Cockburn F, Belton NR, Purvis RJ, Giles MM, Brown JK, Turner TL, et al. Maternal vitamin D intake and mineral metabolism in mothers and their newborn infants. BMJ 1980;281:11-4.
4. Altman DG. Practical statistics for medical research. London: Chapman and Hall, 1991:160-7.
5. Collaborative Group on Antenatal Steroid Therapy. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87. [Medline]