Problems related to selection bias, misclassification bias, statistical analysis, and the chosen population.
Blair et al. investigated the risk of hazardous co-sleeping and other
factors resulting from parental risk behaviors including cigarette
smoking, alcohol and drug use. We have identified that this study presents
problems related to selection bias, misclassification bias, statistical
analysis, and the chosen population. We concluded that these problems
have seriously compromised the results reported in the study.
Selection BIAS
The error due to systematic differences in the characteristics of
those who took part of the study and those who did not is a concern.
Available data shows that alcohol is a big problem in the southwest area
of England. In fact, data from the South West Public Health Observatory
(2008) shows that an estimated of three-quarters of a million (728,530) or
23% of South West residents aged 16–64 are hazardous drinkers, and a
further 118,780 (4%) are dependent drinkers.1 Tables 2 and 4 show a
similar prevalence of alcohol use among mothers of SIDS children (25%).
So, the extremely low prevalence of alcohol use among mothers from the
random control group (2%) and from the high-risk control infant group (4%)
is a concern. Also, over a third of mothers (34%) in England smoke before
or during their pregnancy (defined as those women who smoked one year
prior to conception or at any time during their pregnancy).2 Again, we can
see a lower exposure of another risk factor among mothers from the random
control infants group (only 14% of them smoked during pregnancy). It is
important to highlight the fact that the smoking rates during pregnancy
among mothers from the SIDS infants group (59%) is so much higher than the
smoking rates during pregnancy among women from English social classes IV
and V (around 43% and 41% respectively).
The high rates for refusing to participate in the study may have
increased the probability of having self-selection bias. In fact, only
26.77% of the recruited families from the random control infant group (385
out of 1,438 families) and 17.30% of recruited families from the high-risk
infants control group (206 out of 1,191 families) accepted and
participated in the study. Also, the difficulty in recruiting control
families and the failure to interview control families on weekends, when
drug and alcohol use might be more common, could have also introduced
control selection bias that may have resulted in a bias toward the null.
Misclassification Bias
The classification used to categorize a baby as having died of SIDS
is a concern. According to the U.S. Department of Health and Human
Services, SIDS stands for sudden infant death syndrome, a term that
describes the sudden, unexplained death of an infant younger than 1 year
of age. Hence, it is questionable that the death of a baby who died
because a drunken parent rolled on top of him/her, because a couch cushion
suffocated the baby, or because the baby was unintentionally dropped could
be classified as an unexplained death.
Problems with the Statistical Analysis
There are several concerns related to the validity of the research
findings due to the lack of statistical power and multicollinearity.
Blair et al. reported that some statistically non-significant
findings from the bivariate analysis reported study were significant in
the larger study conducted 10 years ago. It is very likely that the
previous study had enough power to detect these associations. On the other
hand, it is very likely that the 2003-06 study could not find these
significant associations due to the inadequate sample size that
compromised the power of the analysis. The presence of cells with no-
observations may have also compromised the results of these bivariate
analyses.
Regarding the logistic regression models reported in tables 2 and 4,
it is concerning that Blair et al. did not check the multicollinearity
between predictors. If cases of severe multicollinearity were present, the
standard errors for the coefficients may have tended to be very large
(inflated), making the estimated logistic regression coefficients highly
unreliable. Also, some important predictors may have become non-
significantly associated with the answer outcome due to the
multicollinearity. Internal validity concerns are accentuated if we take
also into account the fact that some predictors may not have been included
in these logistic regression models due to the bivariate analysis’ lack of
power.
The inclusion of multiple predictors in each logistic regression
model may have further compromised the internal validity of the study.
Table 2 presents the first logistic regression model including 74 SIDS
infants and 86 random control infants that constitutes a total sample of
160 infants. Table 4 presents a second logistic regression model with 74
SIDS infants and 81 high-risk control infants for a total of 155 infants.
Tables 2 and 4 also show that 21 predictors and 2 covariates were included
in the first logistic regression model and 12 predictors and 6 covariates
in the second model. These logistic regression models based on an
inadequate sample size are not able to detect significant associations
between the response variable and the predictor variables. Finally, the
small sample size may have influenced the magnitude of the association
between predictors and the answer variable. Hence, the previously
mentioned selection and misclassification biases together with a weak
association between the answer and the predictor variables could have
influenced the presence of errors.
The chosen population
It is questionable to state that the Blair et al. work has
contributed to the scientific literature by including a sample of
unexpected infant deaths from birth to age 2 years. First, their sample
age distribution of the SIDS infants was significantly different from
previous studies (more than three weeks younger). Second, based on the
information presented in figure 2, it can be concluded that no infants
more than 52 weeks old (one year) were included in the random control
infants group and the high risk control infants group. Only around 4-5% of
SIDS infants were more than 52 weeks old (n=3 or 4). As a result, Blair et
al.’s conclusions that are generalized for an infant group from birth to
age 2 years lack of validity.
2. Pollard, Becky & Cooke, Helen. Smoking in the South West of
England Department of Public Health, NHS South West Regional Office. South
West Public Health Observatory. Available at: www.swpho.nhs.uk/resource/view.aspx?RID=9136
3. Harrell, F. E. Regression modelling strategies for improved
prognostic prediction. Statistics in Medicine 3: 143-152, 1984.
Competing interests:
None declared
Competing interests:
No competing interests
04 December 2009
Mariano Juan Kanamori Nishimura
PhD student
University of Maryland College Park, School of Public Health. Department of Biostatistics and Epidem
Rapid Response:
Problems related to selection bias, misclassification bias, statistical analysis, and the chosen population.
Blair et al. investigated the risk of hazardous co-sleeping and other
factors resulting from parental risk behaviors including cigarette
smoking, alcohol and drug use. We have identified that this study presents
problems related to selection bias, misclassification bias, statistical
analysis, and the chosen population. We concluded that these problems
have seriously compromised the results reported in the study.
Selection BIAS
The error due to systematic differences in the characteristics of
those who took part of the study and those who did not is a concern.
Available data shows that alcohol is a big problem in the southwest area
of England. In fact, data from the South West Public Health Observatory
(2008) shows that an estimated of three-quarters of a million (728,530) or
23% of South West residents aged 16–64 are hazardous drinkers, and a
further 118,780 (4%) are dependent drinkers.1 Tables 2 and 4 show a
similar prevalence of alcohol use among mothers of SIDS children (25%).
So, the extremely low prevalence of alcohol use among mothers from the
random control group (2%) and from the high-risk control infant group (4%)
is a concern. Also, over a third of mothers (34%) in England smoke before
or during their pregnancy (defined as those women who smoked one year
prior to conception or at any time during their pregnancy).2 Again, we can
see a lower exposure of another risk factor among mothers from the random
control infants group (only 14% of them smoked during pregnancy). It is
important to highlight the fact that the smoking rates during pregnancy
among mothers from the SIDS infants group (59%) is so much higher than the
smoking rates during pregnancy among women from English social classes IV
and V (around 43% and 41% respectively).
The high rates for refusing to participate in the study may have
increased the probability of having self-selection bias. In fact, only
26.77% of the recruited families from the random control infant group (385
out of 1,438 families) and 17.30% of recruited families from the high-risk
infants control group (206 out of 1,191 families) accepted and
participated in the study. Also, the difficulty in recruiting control
families and the failure to interview control families on weekends, when
drug and alcohol use might be more common, could have also introduced
control selection bias that may have resulted in a bias toward the null.
Misclassification Bias
The classification used to categorize a baby as having died of SIDS
is a concern. According to the U.S. Department of Health and Human
Services, SIDS stands for sudden infant death syndrome, a term that
describes the sudden, unexplained death of an infant younger than 1 year
of age. Hence, it is questionable that the death of a baby who died
because a drunken parent rolled on top of him/her, because a couch cushion
suffocated the baby, or because the baby was unintentionally dropped could
be classified as an unexplained death.
Problems with the Statistical Analysis
There are several concerns related to the validity of the research
findings due to the lack of statistical power and multicollinearity.
Blair et al. reported that some statistically non-significant
findings from the bivariate analysis reported study were significant in
the larger study conducted 10 years ago. It is very likely that the
previous study had enough power to detect these associations. On the other
hand, it is very likely that the 2003-06 study could not find these
significant associations due to the inadequate sample size that
compromised the power of the analysis. The presence of cells with no-
observations may have also compromised the results of these bivariate
analyses.
Regarding the logistic regression models reported in tables 2 and 4,
it is concerning that Blair et al. did not check the multicollinearity
between predictors. If cases of severe multicollinearity were present, the
standard errors for the coefficients may have tended to be very large
(inflated), making the estimated logistic regression coefficients highly
unreliable. Also, some important predictors may have become non-
significantly associated with the answer outcome due to the
multicollinearity. Internal validity concerns are accentuated if we take
also into account the fact that some predictors may not have been included
in these logistic regression models due to the bivariate analysis’ lack of
power.
The inclusion of multiple predictors in each logistic regression
model may have further compromised the internal validity of the study.
Table 2 presents the first logistic regression model including 74 SIDS
infants and 86 random control infants that constitutes a total sample of
160 infants. Table 4 presents a second logistic regression model with 74
SIDS infants and 81 high-risk control infants for a total of 155 infants.
Tables 2 and 4 also show that 21 predictors and 2 covariates were included
in the first logistic regression model and 12 predictors and 6 covariates
in the second model. These logistic regression models based on an
inadequate sample size are not able to detect significant associations
between the response variable and the predictor variables. Finally, the
small sample size may have influenced the magnitude of the association
between predictors and the answer variable. Hence, the previously
mentioned selection and misclassification biases together with a weak
association between the answer and the predictor variables could have
influenced the presence of errors.
The chosen population
It is questionable to state that the Blair et al. work has
contributed to the scientific literature by including a sample of
unexpected infant deaths from birth to age 2 years. First, their sample
age distribution of the SIDS infants was significantly different from
previous studies (more than three weeks younger). Second, based on the
information presented in figure 2, it can be concluded that no infants
more than 52 weeks old (one year) were included in the random control
infants group and the high risk control infants group. Only around 4-5% of
SIDS infants were more than 52 weeks old (n=3 or 4). As a result, Blair et
al.’s conclusions that are generalized for an infant group from birth to
age 2 years lack of validity.
References:
1. Walsh, Alice. Calling Time. Reducing alcohol harm in the South
West– a blueprint for joint action. South West Public Health Observatory
2008. Available at
http://www.alcohollearningcentre.org.uk/_library/Calling_Time_Exec_Summa...
2. Pollard, Becky & Cooke, Helen. Smoking in the South West of
England Department of Public Health, NHS South West Regional Office. South
West Public Health Observatory. Available at:
www.swpho.nhs.uk/resource/view.aspx?RID=9136
3. Harrell, F. E. Regression modelling strategies for improved
prognostic prediction. Statistics in Medicine 3: 143-152, 1984.
Competing interests:
None declared
Competing interests: No competing interests