Short communicationA simulation study comparing methods for calculating confidence intervals for directly standardized rates
Introduction
Standardization is an elemental tool used to control the effects of extraneous variables when making a comparison of different rates. A common example is the adjustment of incidence and death rates for the discrepancies in the age structure between populations, since both mortality and morbidity depend strongly on age. Another example occurs in meta-analysis requiring aggregation of outcome rates from several different studies, using weights related to the numbers of subjects in each study. To make statistical inferences about these directly standardized rates (DSRs) it is assumed that they are weighted sums of Poisson parameters and the number of events observed in each group is independent. The population DSRs are a function of many parameters (the set of Poisson rates) and therefore it is not straightforward to calculate the confidence interval on the DSR. In lieu of this, one often assumes the normal distribution and obtains the standard normal confidence intervals.
Simulation studies (Dobson et al., 1991, Swift, 1995) have shown that using the standard normal approach may be inadequate. Recently there have been several methods proposed to improve upon the standard confidence intervals. These include the moment matching method proposed by Dobson et al. (1991); the approximate bootstrap confidence (ABC) method of DiCiccio and Efron (1992) as applied by Swift (1995); and methods based on gamma confidence intervals (Fay and Feuer, 1997, Tiwari et al., 2006), and beta approximations (Tiwari et al., 2006).
The main purpose of this article is to respond to two recent reviews of confidence interval estimation for age-adjusted rates. Ng et al. (2008) considered the performances of 20 different confidence interval construction procedures for direct standardization of rates. They did not consider the ABC method in their review. Tiwari et al. (2006) gave the formulae for the ABC method in an appendix but did not present any further analysis for the method.
Fay and Feuer (1997) found that the upper confidence limit of the ABC and Dobson et al. (1991) methods tended to increase with higher variance of the weights while the upper limit of the gamma method decreased with higher variance of the weights . Ng et al. (2008) recommend the Dobson et al. (1991) moment matching method using an approximation to the mid- confidence interval (labeled M8 by Ng et al. (2008)), if all weights are equal or when some weights are equal and the rest are zero. For small values of , Ng et al. (2008) again recommend M8 as the best alternative when both a lower coverage probability and interval length are of interest, as well as a method proposed by Tiwari et al. (2006) based on the gamma distribution (G4) when the coverage probabilities of the confidence intervals are preferred to be above the nominal level. They recommend the method based on a gamma distribution (G1) for moderate and large values of .
I reproduce some of Ng et al.’s (2008) random simulation results for their recommended methods to compare with the ABC results. Exact replication is not possible without knowing the starting seed numbers they used in the random generators for their study. Monte Carlo errors are presented in the discussion section of the article to estimate the differences one could expect to see for different repetitions of the study. In Section 2, I first give the background on the notation for calculating directly standardized rates and then briefly present the confidence interval formulae for Fay and Feuer’s (1997) gamma interval [G1]; Tiwari et al.’s (2006) less conservative gamma method [G4]; Swift’s (1995) ABC intervals [ABC]; and Dobson et al. (1991) moment matching method using an approximation to the mid- Poisson confidence interval [M8]. In Section 3, I calculate the 95% ABC coverage properties using a similar random scenario used by Ng et al. (2008) for a simulation of 500 samples each. I use 10,000 replications for each sample to decrease the magnitude of differences from repeated simulations. In Section 4, I compare the performance of each confidence interval method in terms of their coverage probabilities and their symmetry of left and right non-coverage using graphical displays for the results of all 500 simulations as a function of . I also present a table showing the average expected widths for 3 different ranges of .
Section snippets
Confidence intervals for directly standardized rates
The discussion of calculating confidence intervals of the weighted sum of Poisson parameters will be drawn from the example of directly age-adjusted rates. The observed directly age-adjusted rate is defined as , where the represent the number of incident cases (or other outcome events) in age groups and with the known constants (for example, Segi World Standard Population) and the number of person-years observed for age group . The general assumption
Monte Carlo simulation study
In this section I conduct a Monte Carlo simulation for each of the 4 confidence interval methods consisting of 500 simulations of 10,000 replications each. The simulation study was performed using SAS®Version 9.1.3 software. I let both the weights and the true means () be independent random samples drawn from a uniform distribution on (0, 1) for each simulation. I chose the number of cells and standardized the true means such that and similarly standardized the weights to
Results
Fig. 1 shows the estimated coverage probabilities of the 95% confidence intervals computed by the 4 methods and plotted by the variance of the weights used for each random simulation. There are three horizontal reference lines drawn: a dashed line showing the nominal value expected for a 95% confidence level, another dashed line drawn at the 96% line showing the lower limit for calculating PC and a dashed line drawn to show the upper limit used in calculating PL. Ideally all the simulation
Discussion
The choice of which confidence interval procedure to use depends as much on the statistician as the actual properties of the interval. Comparison of the performance of the confidence intervals is conducted in terms of their: (1) coverage properties; (2) symmetry of left and right non-coverage and (3) expected confidence width. Some statisticians want conservative intervals which guarantee coverage to be at or above the nominal level. To achieve this they sacrifice the widths of the confidence
References (11)
- et al.
Confidence interval estimating procedures for standardized incidence rates
Computational Statistics and Data Analysis
(2008) - et al.
Mid-P confidence intervals for the Poisson expectation
Statistics in Medicine
(1994) - et al.
More accurate confidence intervals in exponential families
Biometrika
(1992) - et al.
Confidence intervals for weighted sums of Poisson parameters
Statistics in Medicine
(1991) - et al.
An introduction to the bootstrap
(1993)
Cited by (9)
Behavioral event detection and rate estimation for autonomous vehicle evaluation
2023, Applied Stochastic Models in Business and IndustryINDEX FOR COMPARING INTERVAL ESTIMATION METHODOLOGIES FOR DIRECT STANDARDIZED RATES
2021, Revista de la Facultad de CienciasModified large sample confidence intervals for Poisson distributions: Ratio, weighted average, and product of means
2016, Communications in Statistics - Theory and Methods