Intended for healthcare professionals

CCBYNC Open access
Research

Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review

BMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i5170 (Published 13 October 2016) Cite this as: BMJ 2016;355:i5170

Re: Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review

Higgins et al report that diphtheria-tetanus-pertussis (DTP) vaccine was associated with a 38% (-8% to 108%) increase in all-cause mortality in a random effects model (I-squared = 71%) [1]. They concede that “The mortality rate was very high among unvaccinated children in the Papua New Guinea (PNG) study [2], and two referees had notable concerns about this study. Excluding it from the meta-analysis gave a relative risk of 1.36 (1.09 to 1.68).” However, they have used a fixed effects model for this estimate, which is inappropriate given that I-squared is 68%. The correct estimate using a random effects model is a 53% (2% to 130%) increase in mortality.

Two of the three referees had “notable concerns” about the PNG study because it had very severe frailty bias, survival bias, and vaccine-status bias. The bias in the PNG study is clearly seen in Figure 8 of the review with a marked “shift to the left” of the PNG estimates for both BCG and DTP compared to the other studies [1].

Frailty bias occurred because vaccines were available only at clinics, and clinics were instructed not to vaccinate sick children. Unvaccinated children were not given BCG if they were sick, and BCG-vaccinated children were not given DTP if they were sick. Consequently, mortality was extraordinarily high at 233 per 1000 person-years in unvaccinated children aged 1-5 months in the study, compared to only 31 per 1000 person-years in vaccinated children. In addition, assuming at least 50% of infant deaths occur in neonates, mortality was <34 per 1000 live births in children aged 1-11 months in the region at the time of the study, and <60 per 1000 live births for age 1-11 months in the region in the 1920s before vaccines and antibiotics were available [3]. So mortality in unvaccinated children was 7.5 times higher than in vaccinated children, 6.9 times higher than in the general population at the time, and 3.9 times higher than 60-70 years earlier (before vaccines and antibiotics were available).

Survival bias occurs if a dead child who is classified as unvaccinated or BCG-vaccinated has actually received DTP. The PNG paper says [2], “If there was no clinic card linked to a child on the demographic database, it was concluded [assumed] that the child had not been vaccinated” - and because no child was excluded for lack of information, the analysis assumed that the data were perfect. However the authors admit that their data were *not* perfect, “Families of children who had no record of having received Pnc PS [pneumococcal polysaccharide] vaccine by age 2 years were visited in order to record any missed Pnc PS vaccinations but health books were not checked for other immunizations. Unfortunately, the way results from these home visits were recorded does not provide satisfactory information on quality of our routine immunization data collection. Similarly, if a child died, we made a point of trying to determine Pnc PS immunization status. It is therefore likely that the most reliable data in this study are for Pnc PS vaccine.” That is, the data for BCG and DTP are *not* reliable (including the immunisation status of dead children), and yet the analysis assumes perfect data.

Vaccination status bias was present in the PNG study because only 76% of the DTP group received the required sequence of DTP after BCG [2]; 24% received BCG with or after DTP, which reverses the harm from DTP (Higgins et al, Figure 5). In addition, pigbel vaccine was given routinely with DTP, and the DTP hypothesis mandates exclusion if there was “any other vaccine given with any dose of DTP (other than OPV)” [4].

Because of very severe frailty bias, survival bias and vaccination status bias, the PNG study should have been classified as very high risk of bias and excluded from the analysis of the effect of DTP on all-cause mortality. In addition, Higgins et al used the wrong model to test the effect of DTP with the PNG study excluded; the correct analysis uses a random effects model, which shows that DTP is associated with a 53% (2% to 130%) increase in mortality. As I have suggested previously, if the PNG, Bangladesh and Burkina Faso studies are all excluded because of bias, DTP is associated with a 91% (46% to 150%) increase in all-cause mortality in children in high-mortality communities [5].

1 Higgins JPT, Soares-Weiser K, López-López JA, et al. Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review. BMJ 2016;355:i5170.
2 Lehmann D, Vail J, Firth MJ, et al. Benefits of routine immunizations on childhood survival in Tari, Southern Highlands Province, Papua New Guinea. Int J Epidemiol 2005;34:138–48.
3 Lehmann D. Demography and causes of death among the Huli in the Tari Basin. P N G Med J 2002;45:51–62.
4 Fine PEM, Smith PG. ‘Non-specific effects of vaccines’--an important analytical insight, and call for a workshop. Trop Med Int Health 2007;12:1–4.
5 Shann F. Re: Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review. http://www.bmj.com/content/355/bmj.i5170/rr-0

Competing interests: No competing interests

14 December 2016
Frank Shann
Professorial Fellow
Department of Paediatrics, University of Melbourne
Parkville, Victoria 3052, Australia