After publication of the Utrecht study on perinatal mortality [1],
several criticisms of the methodology of the study were published in the
BMJ as well as in general newspapers in the Netherlands. The Dutch medical
journal 'Medisch Contact' interviewed Jan Vandenbroucke, Professor of
Clinical Epidemiology at Leiden university, as an outside methodologist
about these criticisms [2]. This rapid response is an English language
version of this interview, recast for an international audience. According
to Vandenbroucke the criticisms do not detract from the study results.
Unless a major calculation error was made, or unless the results are a
mere play of chance, the conclusion is justified that the mortality in
low risk deliveries, started under the supervision of midwives, is higher
than in high risk deliveries started under the supervision of
obstetricians.
Criticism: the infant mortality rate in the low risk of the Utrecht
study differs from the figures in the national TNO [3] study. It can
therefore not be right.
Vandenbroucke: "The fact is - the mortality rates don't differ that
much at all. The Dutch figures from the Peristat II survey show about 2,8
deaths per 1000 births, after subtraction of the mortality by congenital
disorders, and that's almost the same as the overall mortality rate found
by Utrecht, which is 2,62 deaths per 1000 deliveries.
Mortality rates in the low risk groups differ between the TNO study
and the Utrecht study. That's rather obvious, because the composition of
the low-risk groups is also different. The TNO researchers compared
intended home births with intended outpatient deliveries. In both cases
the midwife was in charge of the delivery, so the TNO study looked at the
planned place of birth, home or outpatient department, and found no
difference. Therefore the Utrecht researches were justified to consider
these births in one group as midwife-led births and compare them to births
under supervision of the obstretician. The latter are by definition high
risk births in the Dutch situation, because only women whose delivery is
considered as high risk by midwives are referred to an obstetrician,
either during pregnancy or acutely during delivery. Thus, the Utrecht
study contrasted different care givers.
A second difference between the studies is that the TNO study looks
at where the birth was planned, being unclear about what time during
pregnancy the planning of the birth was recorded in the database, while
the Utrecht study looked at whether the delivery was actually started
under supervision of the midwife or the obstetrician. This means that up
to the beginning of labour, the women who started their delivery under
supervision of a midwife were still regarded as low-risk, otherwise the
woman would have been referred to an obstetrician.
In addition, there appear to be more exclusions in the TNO study than
in the Utrecht study. All these differences make the two studies
incomparable in terms of design."
Criticism: The analysis of the Utrecht study has no correction for
confounding factors.
Vandenbroucke: "If the Utrecht researchers had been able to correct
for confounders, the difference would have been even larger. Because this
is a high-versus low-risk selection, you know which direction the
confounding factors operate: they go against the direction of the results
that were found.
The Utrecht researchers found less infant mortality when delivery
started with the obstetrician than if it started under supervision of the
midwife, although obstetricians see women with a higher risk. This is
unprecedented in medical research. Compare it with cardiovascular disease:
a fifty year old man with high cholesterol, hypertension and myocardial
infarction can be treated for all these risk factors, but his risk will
never be as low as that of a man of the same age who has no risk factors
whatsoever and never had a myocardial infraction - let alone that his risk
would become lower. Yet that is what was found in the Utrecht study."
Criticism: the Utrecht researchers could not adequately calculate the
denominator because of uncertainties in the postal code delimitation of
the area of study. This makes the difference in mortality larger than it
is.
Vandenbroucke: "About such criticism you can do a sensitivity
analysis. Imagine that the Utrecht researchers underestimated the number
of births with midwives. Imagine that the number of births with midwives
was actually 50% to two times larger. Then the infant mortality rate would
still be higher with the midwives. Only if the denominator had been three
times underestimated one would find that the deliveries by midwives have
fewer deaths - which is what you would actually expect in a low risk
group. The criticism that some 5 percent of women from the edges of the
postcode area would have been missed does not really invalidated the
results. Obviously it can not be excluded that the Utrecht researchers
made a gross calculation error or that by mere play of chance the
mortality with the midwives was too high. Still, the overall perinatal
mortality in the study corresponds to the national average. Now, you can
continue to stare at these data and devise all kinds of explanations, but
"the best (re)analysis of a study is to repeat it" quoting Cornfield [4].
This must be possible in other regions of the Netherlands with existing
data."
Criticism; the number of NICU admissions is no different between the
two groups in the Utrecht study. This is strange, because in the group
with more deaths you expect also more severe disease.
Vandenbroucke: "This is not easy to explain. However, the rate of
NICU admissions in the Utrecht study was highest when the mother was
transferred to the obstetrician during the delivery. This is in line with
the overall result that mortality is highest in the group transferred
during delivery. After start of delivery with a midwife 30 percent of
women are transferred to the obstetrician, and in primigravida the figure
is as high as 50 percent. Presumably, this is the cause of the problem:
prediction of which woman will need referral does not seem to work well."
Criticism: the study was done in an existing database and the
protocol did not originally specify this analysis. And, even if stated in
the BMJ title, this is not a prospective study.
Vandenbroucke: "There is much debate about the use of the word
prospective [5, see item 4 in reference 6]. According to one of the most
classic epidemiological definitions, which is seemingly also used by the
BMJ, this research is called 'prospective' as a synonym for a cohort
study. As an important safeguard against bias, the data about the
original caregiver (where the delivery started) were fixed before the
outcome was known. It doesn't matter that there was no protocol in advance
for these findings: data are data. Indeed, it would be a disgrace to
ignore such data. Compare it to an unexpected side effect of a drug: this
must be reported even if it happens to be found in a study with another
aim. "
Criticism: deliveries supervised by a obstetrician lead to more
medical procedures, with all the disadvantages of increased
medicalization.
Vandenbroucke: "This research shows that medical interventions like
those that happened in Utrecht made sense, as there is less perinatal
mortality in deliveries under the supervision of the obstetrician. It is
evident that the problem lies in the transition from midwife to
obstetrician: that's where the mortality rate is especially high. A new
national research program intends to lower the barriers between midwives
and obstetrician, an aim that was specified in the conclusions of a 2010
report on Pregnancy and Child Birth on behalf of the Health Ministry of
The Netherlands. Already in the 1970s and 80s it was reported that the
infant mortality in the Netherlands, although decreasing, decreased slower
than in other European countries [7]. At present we know that perinatal
mortality in the Netherlands is about twice that in leading European
countries. Apparently it takes decades before professionals start
collaborating on the basis of research data."
Jan P Vandenbroucke, Professor of Clinical Epidemiology, Leiden
University Medical Center, and Heleen Croonen, Medical Journalist, Medisch
Contact, the Netherlands
[1] Evers AC, Brouwers HA, Hukkelhoven CW, Nikkels PG, Boon J, van
Egmond-Linden A, Hillegersberg J, Snuif YS, Sterken-Hooisma S, Bruinse HW,
Kwee A. Perinatal mortality and severe morbidity in low and high risk term
pregnancies in the Netherlands: prospective cohort study. BMJ. 2010 Nov
2;341:c5639. doi: 10.1136/bmj.c5639.
[3] de Jonge A, van der Goes BY, Ravelli AC, Amelink-Verburg MP, Mol
BW, Nijhuis JG, Bennebroek Gravenhorst J, Buitendijk SE. Perinatal
mortality and morbidity in a nationwide cohort of 529,688 low-risk planned
home and hospital births. BJOG. 2009;116:1177-84.
[4] Cornfield J. The University Group Diabetes Program. A further
statistical analysis of the mortality findings. JAMA 1971;217:1676-87.
[5] Vandenbroucke JP. Prospective or retrospective: what's in a name?
BMJ. 1991;302:249-50.
[6] Vandenbroucke JP, von Elm E, Altman DG, G?tzsche PC, Mulrow CD,
Pocock SJ, Poole C, Schlesselman JJ, Egger M; STROBE Initiative.
Strengthening the Reporting of Observational Studies in Epidemiology
(STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297.
[7] Hoogendoorn D. Indrukwekkende en tegelijk teleurstellende daling
van de perinatale sterfte in Nederland. [ [Impressive but still
disappointing decline in perinatal mortality in The Netherlands] [Article
in Dutch] Ned Tijdschr Geneeskd 1986; 130: 1436-440.
Rapid Response:
UMC Utrecht did a good job
After publication of the Utrecht study on perinatal mortality [1],
several criticisms of the methodology of the study were published in the
BMJ as well as in general newspapers in the Netherlands. The Dutch medical
journal 'Medisch Contact' interviewed Jan Vandenbroucke, Professor of
Clinical Epidemiology at Leiden university, as an outside methodologist
about these criticisms [2]. This rapid response is an English language
version of this interview, recast for an international audience. According
to Vandenbroucke the criticisms do not detract from the study results.
Unless a major calculation error was made, or unless the results are a
mere play of chance, the conclusion is justified that the mortality in
low risk deliveries, started under the supervision of midwives, is higher
than in high risk deliveries started under the supervision of
obstetricians.
Criticism: the infant mortality rate in the low risk of the Utrecht
study differs from the figures in the national TNO [3] study. It can
therefore not be right.
Vandenbroucke: "The fact is - the mortality rates don't differ that
much at all. The Dutch figures from the Peristat II survey show about 2,8
deaths per 1000 births, after subtraction of the mortality by congenital
disorders, and that's almost the same as the overall mortality rate found
by Utrecht, which is 2,62 deaths per 1000 deliveries.
Mortality rates in the low risk groups differ between the TNO study
and the Utrecht study. That's rather obvious, because the composition of
the low-risk groups is also different. The TNO researchers compared
intended home births with intended outpatient deliveries. In both cases
the midwife was in charge of the delivery, so the TNO study looked at the
planned place of birth, home or outpatient department, and found no
difference. Therefore the Utrecht researches were justified to consider
these births in one group as midwife-led births and compare them to births
under supervision of the obstretician. The latter are by definition high
risk births in the Dutch situation, because only women whose delivery is
considered as high risk by midwives are referred to an obstetrician,
either during pregnancy or acutely during delivery. Thus, the Utrecht
study contrasted different care givers.
A second difference between the studies is that the TNO study looks
at where the birth was planned, being unclear about what time during
pregnancy the planning of the birth was recorded in the database, while
the Utrecht study looked at whether the delivery was actually started
under supervision of the midwife or the obstetrician. This means that up
to the beginning of labour, the women who started their delivery under
supervision of a midwife were still regarded as low-risk, otherwise the
woman would have been referred to an obstetrician.
In addition, there appear to be more exclusions in the TNO study than
in the Utrecht study. All these differences make the two studies
incomparable in terms of design."
Criticism: The analysis of the Utrecht study has no correction for
confounding factors.
Vandenbroucke: "If the Utrecht researchers had been able to correct
for confounders, the difference would have been even larger. Because this
is a high-versus low-risk selection, you know which direction the
confounding factors operate: they go against the direction of the results
that were found.
The Utrecht researchers found less infant mortality when delivery
started with the obstetrician than if it started under supervision of the
midwife, although obstetricians see women with a higher risk. This is
unprecedented in medical research. Compare it with cardiovascular disease:
a fifty year old man with high cholesterol, hypertension and myocardial
infarction can be treated for all these risk factors, but his risk will
never be as low as that of a man of the same age who has no risk factors
whatsoever and never had a myocardial infraction - let alone that his risk
would become lower. Yet that is what was found in the Utrecht study."
Criticism: the Utrecht researchers could not adequately calculate the
denominator because of uncertainties in the postal code delimitation of
the area of study. This makes the difference in mortality larger than it
is.
Vandenbroucke: "About such criticism you can do a sensitivity
analysis. Imagine that the Utrecht researchers underestimated the number
of births with midwives. Imagine that the number of births with midwives
was actually 50% to two times larger. Then the infant mortality rate would
still be higher with the midwives. Only if the denominator had been three
times underestimated one would find that the deliveries by midwives have
fewer deaths - which is what you would actually expect in a low risk
group. The criticism that some 5 percent of women from the edges of the
postcode area would have been missed does not really invalidated the
results. Obviously it can not be excluded that the Utrecht researchers
made a gross calculation error or that by mere play of chance the
mortality with the midwives was too high. Still, the overall perinatal
mortality in the study corresponds to the national average. Now, you can
continue to stare at these data and devise all kinds of explanations, but
"the best (re)analysis of a study is to repeat it" quoting Cornfield [4].
This must be possible in other regions of the Netherlands with existing
data."
Criticism; the number of NICU admissions is no different between the
two groups in the Utrecht study. This is strange, because in the group
with more deaths you expect also more severe disease.
Vandenbroucke: "This is not easy to explain. However, the rate of
NICU admissions in the Utrecht study was highest when the mother was
transferred to the obstetrician during the delivery. This is in line with
the overall result that mortality is highest in the group transferred
during delivery. After start of delivery with a midwife 30 percent of
women are transferred to the obstetrician, and in primigravida the figure
is as high as 50 percent. Presumably, this is the cause of the problem:
prediction of which woman will need referral does not seem to work well."
Criticism: the study was done in an existing database and the
protocol did not originally specify this analysis. And, even if stated in
the BMJ title, this is not a prospective study.
Vandenbroucke: "There is much debate about the use of the word
prospective [5, see item 4 in reference 6]. According to one of the most
classic epidemiological definitions, which is seemingly also used by the
BMJ, this research is called 'prospective' as a synonym for a cohort
study. As an important safeguard against bias, the data about the
original caregiver (where the delivery started) were fixed before the
outcome was known. It doesn't matter that there was no protocol in advance
for these findings: data are data. Indeed, it would be a disgrace to
ignore such data. Compare it to an unexpected side effect of a drug: this
must be reported even if it happens to be found in a study with another
aim. "
Criticism: deliveries supervised by a obstetrician lead to more
medical procedures, with all the disadvantages of increased
medicalization.
Vandenbroucke: "This research shows that medical interventions like
those that happened in Utrecht made sense, as there is less perinatal
mortality in deliveries under the supervision of the obstetrician. It is
evident that the problem lies in the transition from midwife to
obstetrician: that's where the mortality rate is especially high. A new
national research program intends to lower the barriers between midwives
and obstetrician, an aim that was specified in the conclusions of a 2010
report on Pregnancy and Child Birth on behalf of the Health Ministry of
The Netherlands. Already in the 1970s and 80s it was reported that the
infant mortality in the Netherlands, although decreasing, decreased slower
than in other European countries [7]. At present we know that perinatal
mortality in the Netherlands is about twice that in leading European
countries. Apparently it takes decades before professionals start
collaborating on the basis of research data."
Jan P Vandenbroucke, Professor of Clinical Epidemiology, Leiden
University Medical Center, and Heleen Croonen, Medical Journalist, Medisch
Contact, the Netherlands
[1] Evers AC, Brouwers HA, Hukkelhoven CW, Nikkels PG, Boon J, van
Egmond-Linden A, Hillegersberg J, Snuif YS, Sterken-Hooisma S, Bruinse HW,
Kwee A. Perinatal mortality and severe morbidity in low and high risk term
pregnancies in the Netherlands: prospective cohort study. BMJ. 2010 Nov
2;341:c5639. doi: 10.1136/bmj.c5639.
[2] Croonen HF. Minder babysterfte bij gynecoloog. Medisch Contact
2010 http://medischcontact.artsennet.nl/blad/Tijdschriftartikel/Minder-
babysterfte-bij-gynaecoloog.htm
[3] de Jonge A, van der Goes BY, Ravelli AC, Amelink-Verburg MP, Mol
BW, Nijhuis JG, Bennebroek Gravenhorst J, Buitendijk SE. Perinatal
mortality and morbidity in a nationwide cohort of 529,688 low-risk planned
home and hospital births. BJOG. 2009;116:1177-84.
[4] Cornfield J. The University Group Diabetes Program. A further
statistical analysis of the mortality findings. JAMA 1971;217:1676-87.
[5] Vandenbroucke JP. Prospective or retrospective: what's in a name?
BMJ. 1991;302:249-50.
[6] Vandenbroucke JP, von Elm E, Altman DG, G?tzsche PC, Mulrow CD,
Pocock SJ, Poole C, Schlesselman JJ, Egger M; STROBE Initiative.
Strengthening the Reporting of Observational Studies in Epidemiology
(STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297.
[7] Hoogendoorn D. Indrukwekkende en tegelijk teleurstellende daling
van de perinatale sterfte in Nederland. [ [Impressive but still
disappointing decline in perinatal mortality in The Netherlands] [Article
in Dutch] Ned Tijdschr Geneeskd 1986; 130: 1436-440.
Competing interests: No competing interests