Short communication
A simulation study comparing methods for calculating confidence intervals for directly standardized rates

https://doi.org/10.1016/j.csda.2009.10.021Get rights and content

Abstract

This study provides a supplemental report of the performance of the confidence interval for direct standardized rates obtained by the approximate bootstrap method (ABC) method. The ABC method was not considered by the Ng et al. (2008) paper which compared different methods of interval construction. A graphical comparison of the coverage probability and the ratio of the right to left non-coverage probabilities, as a function of the variance in the weights used for each simulation point, are given for the ABC method, as well as three of the recommended procedures by Ng et al. The expected confidence interval lengths are also reported. The ABC intervals are shown to be good competitors compared with the other three confidence intervals.

Introduction

Standardization is an elemental tool used to control the effects of extraneous variables when making a comparison of different rates. A common example is the adjustment of incidence and death rates for the discrepancies in the age structure between populations, since both mortality and morbidity depend strongly on age. Another example occurs in meta-analysis requiring aggregation of outcome rates from several different studies, using weights related to the numbers of subjects in each study. To make statistical inferences about these directly standardized rates (DSRs) it is assumed that they are weighted sums of Poisson parameters and the number of events observed in each group is independent. The population DSRs are a function of many parameters (the set of Poisson rates) and therefore it is not straightforward to calculate the confidence interval on the DSR. In lieu of this, one often assumes the normal distribution and obtains the standard normal confidence intervals.

Simulation studies (Dobson et al., 1991, Swift, 1995) have shown that using the standard normal approach may be inadequate. Recently there have been several methods proposed to improve upon the standard confidence intervals. These include the moment matching method proposed by Dobson et al. (1991); the approximate bootstrap confidence (ABC) method of DiCiccio and Efron (1992) as applied by Swift (1995); and methods based on gamma confidence intervals (Fay and Feuer, 1997, Tiwari et al., 2006), and beta approximations (Tiwari et al., 2006).

The main purpose of this article is to respond to two recent reviews of confidence interval estimation for age-adjusted rates. Ng et al. (2008) considered the performances of 20 different confidence interval construction procedures for direct standardization of rates. They did not consider the ABC method in their review. Tiwari et al. (2006) gave the formulae for the ABC method in an appendix but did not present any further analysis for the method.

Fay and Feuer (1997) found that the upper confidence limit of the ABC and Dobson et al. (1991) methods tended to increase with higher variance of the weights while the upper limit of the gamma method decreased with higher variance of the weights (var(wi)). Ng et al. (2008) recommend the Dobson et al. (1991) moment matching method using an approximation to the mid-p confidence interval (labeled M8 by Ng et al. (2008)), if all weights are equal or when some weights are equal and the rest are zero. For small values of var(wi), Ng et al. (2008) again recommend M8 as the best alternative when both a lower coverage probability and interval length are of interest, as well as a method proposed by Tiwari et al. (2006) based on the gamma distribution (G4) when the coverage probabilities of the confidence intervals are preferred to be above the nominal level. They recommend the method based on a gamma distribution (G1) for moderate and large values of var(wi).

I reproduce some of Ng et al.’s (2008) random simulation results for their recommended methods to compare with the ABC results. Exact replication is not possible without knowing the starting seed numbers they used in the random generators for their study. Monte Carlo errors are presented in the discussion section of the article to estimate the differences one could expect to see for different repetitions of the study. In Section 2, I first give the background on the notation for calculating directly standardized rates and then briefly present the confidence interval formulae for Fay and Feuer’s (1997) gamma interval [G1]; Tiwari et al.’s (2006) less conservative gamma method [G4]; Swift’s (1995) ABC intervals [ABC]; and Dobson et al. (1991) moment matching method using an approximation to the mid-p Poisson confidence interval [M8]. In Section 3, I calculate the 95% ABC coverage properties using a similar random scenario used by Ng et al. (2008) for a simulation of 500 samples each. I use 10,000 replications for each sample to decrease the magnitude of differences from repeated simulations. In Section 4, I compare the performance of each confidence interval method in terms of their coverage probabilities and their symmetry of left and right non-coverage using graphical displays for the results of all 500 simulations as a function of var(wi). I also present a table showing the average expected widths for 3 different ranges of var(wi).

Section snippets

Confidence intervals for directly standardized rates

The discussion of calculating confidence intervals of the weighted sum of Poisson parameters will be drawn from the example of directly age-adjusted rates. The observed directly age-adjusted rate Rˆ is defined as Rˆ=i=1kwixi, where the xi(i=1,,k) represent the number of incident cases (or other outcome events) in k age groups and wi=ci/Pi with the ci known constants (for example, Segi World Standard Population) and Pi the number of person-years observed for age group i. The general assumption

Monte Carlo simulation study

In this section I conduct a Monte Carlo simulation for each of the 4 confidence interval methods consisting of 500 simulations of 10,000 replications each. The simulation study was performed using SAS®Version 9.1.3 software. I let both the weights (wi) and the true means (θi) be independent random samples drawn from a uniform distribution on (0, 1) for each simulation. I chose the number of cells k=6 and standardized the true means such that i=16θi=20 and similarly standardized the weights to

Results

Fig. 1 shows the estimated coverage probabilities of the 95% confidence intervals computed by the 4 methods and plotted by the variance of the weights used for each random simulation. There are three horizontal reference lines drawn: a dashed line showing the nominal value expected for a 95% confidence level, another dashed line drawn at the 96% line showing the lower limit for calculating PC and a dashed line drawn to show the upper limit used in calculating PL. Ideally all the simulation

Discussion

The choice of which confidence interval procedure to use depends as much on the statistician as the actual properties of the interval. Comparison of the performance of the confidence intervals is conducted in terms of their: (1) coverage properties; (2) symmetry of left and right non-coverage and (3) expected confidence width. Some statisticians want conservative intervals which guarantee coverage to be at or above the nominal level. To achieve this they sacrifice the widths of the confidence

References (11)

  • H.K.T. Ng et al.

    Confidence interval estimating procedures for standardized incidence rates

    Computational Statistics and Data Analysis

    (2008)
  • G.R. Cohen et al.

    Mid-P confidence intervals for the Poisson expectation

    Statistics in Medicine

    (1994)
  • T.J. DiCiccio et al.

    More accurate confidence intervals in exponential families

    Biometrika

    (1992)
  • A.J. Dobson et al.

    Confidence intervals for weighted sums of Poisson parameters

    Statistics in Medicine

    (1991)
  • B. Efron et al.

    An introduction to the bootstrap

    (1993)
There are more references available in the full text version of this article.
View full text