Editorials

Changing practice in maternity care

It's hard to know what works

Papers p 1041

The tenet that clinical practice should be guided by rigorous evidence has become so ingrained that clinicians who are slow on the uptake are seen as not aware of the evidence, bogged down by tradition, orworsehaving selfish motives for ignoring evidence. Rarely is the evidence itself questioned. Yet, if evidence were a straightforward concept, there would be no reason for the two disciplines that appear to be governed by it, law and medicine, to be at loggerheads so often.

The evidence available does not necessarily reveal what you are interested in for a particular situation. Thus many reviews in the Cochrane Library, the gold standard of systematic reviews, devote no attention to adverse effects in assessing the effectiveness of health care interventions (Bastian H, Middleton P. Cochrane Colloquium, Amsterdam, 1997). Yet any intervention (be it advice, screening for disease, drugs, or surgery) that is likely to be beneficial for some people is also likely to harm others. Even if the evidence is clear on the effectiveness of an approach, it does not necessarily reveal how to pursue that approach. For example, systematic reviews may show benefits of antibiotic treatment for preterm prelabour rupture of the membranes, but they do not show what to prescribe and for how long.^1-3

The paper by Wyatt et al in this issue (p 1041), addressing how to enhance the use of evidence, itself demonstrates how "evidence" can fall short of being evidence.⁴ Although this group used evidence's golden tool, the randomised trial, they chose the toss of a coin as the method of randomisation. This process should be secure, but there is good evidence that it is not. ^{5 6} Of the four outcomes addressed, two showed a statistically significant imbalance between intervention and control groups before the trial and two differed significantly in completeness of outcome assessment before or after the trial.

Thus, before the trial, vacuum extraction was used in 36.1% of women in intervention units and in 54.5% in control units (difference 18.2%; 95% confidence interval 11.2% to 25.3%). Appropriate suture material was used in 8.7% of cases in intervention units and 25.1% in control units (difference 16.5%; 11.1% to 21.9%). Assessment of outcome criteria, set at 30 births per unit, was incomplete for sutures at the onset for 3.6% of women in the intervention units and for 9.2% in the control units (difference 5.6%; 2.2% to 9.1%). After the trial it was incomplete for sutures in 5.6% of women in intervention units and in 10.0% in control units (difference 4.4%; 0.6% to 8.0%) and for antibiotic prophylaxis in 12.8% of women in intervention units and 23.8% in control units (difference 11.1%; 5.6% to 16.5%). Thus, there is only one outcome measure (use of corticosteroids) devoid of glaring imbalances in either a priori characteristics or ascertainment, but its assessment relates to no more than three births per participating unit.

People wishing to examine evidence before bowing to its aureolewhich is what pursuit of evidence should promotecan find only one set of data, in figure 3, that is detailed enough to be assessed independently. This figure shows, firstly, the significant difference at baseline between intervention and control units mentioned above. Secondly, 22 of the 25 units had a rate of use of ventouse extraction at baseline that was either at or outside the 95% confidence interval for the average (36% to 55%). Twelve of these units (8 intervention and 4 control) had base rates at or below the 95% range; all had a higher rate at follow up. Of the 10 (3 intervention and 7 control) above the range, all but 2 (1 intervention and 1 control) had lower rates at follow up. Thirdly, of the 25 units, 6 had rates at follow up that differed 10% or less from the base rate: 3 were intervention and 3 were control units. Of the 19 others, 13 (7 intervention and 6 control) were more than 10% higher at follow up and 6 (2 intervention and 4 control) were more than 10% lower. This certainly questions the relevance of the statistically significant increase in the rate of ventouse extraction reported to be associated with the intervention.

Rather, the figure shows that the rate of childbirth interventions can vary considerably from one time to another irrespective of whether or not the people who allegedly control these rates have been made aware of the evidence about these interventions. It also indicates that assessing 30 maternity care procedures per unit is not likely to reflect practice in that unit adequately. This is not surprising as most people would dismiss consecutive series of no more than 30 common procedures, such as operative delivery and episiotomy, as appropriate indicators of practice.

Of course, it would have been surprising if the authors had found a marked effect of their visit to a lead obstetrician and midwife. Indeed, the evidence on the outcomes that they addressed had been available electronically and in well publicised full⁷ and abridged⁸ texts for several years. Lead practitioners who had any serious interest in considering the evidence would surely have sought it out well before this study's intervention. Perhaps it is too simplistic to expect that merely exposing practitioners to evidence will change practicehowever intensive the exposure. Clinical practice changes all the time, but the momentum of change, and what drives it, are poorly understood. For some, change goes too fast, for others too slow, and for those who want to have a significant impact on it, the methods for achieving it are still far from clear.