BMJ  2004;329:1039-1042 (30 October), doi:10.1136/bmj.329.7473.1039

Education and debate

Inappropriate use of randomised trials to evaluate complex phenomena: case study of vaginal breech delivery

¹ Department of Obstetrics and Gynaecology, University of British Columbia, BC Women's Hospital, Vancouver, BC, V6H 3V5 Canada

As randomised trials continue to ascend in the evolution of evidence based medicine, we must recognise and respect their limitations when examining complex phenomena in heterogeneous populations

Introduction

Randomised controlled trials have greatly improved the quality of evidence guiding clinical practice, but when applied to complex phenomena, they have important limitations. Complex patient populations with poorly quantifiable variations between individuals present one area of difficulty; complex procedures requiring skill and clinical judgment present another. A large, well designed, and well executed randomised controlled trial of breech presentation at term, the "term breech trial," by Hannah et al rapidly dictated a new standard of care for the management of breech deliveries around the world.¹ Yet this trial failed to adequately appreciate both the complex nature of vaginal breech delivery and the complex mix of operator variables necessary for its safe conduct. Widespread acceptance of this trial's results has breached the limits of evidence based medicine.

Hannah et al's trial showed a significant increase in perinatal mortality and morbidity in women randomised to a trial of labour compared with elective caesarean section.¹ The trial's methodological flaws have been examined,^2-4 but the intrinsic limitations of applying large scale randomisation to complex phenomena have received little attention. These limitations are the focus of this paper.

Bias of licence

Many of the term breech trial's 121 centres were in North America, where 13% of breech presentations at term were delivered vaginally.⁵ The study achieved a successful vaginal delivery rate of 57% by asking those centres with vaginal birth rates under 40% in the labour group to increase the rate or withdraw from participation.⁶ Individual centres rates of vaginal breech delivery at baseline were not reported, but many would have tripled their vaginal delivery rate overnight.

The vaginal delivery of a breech baby involves risk. Cord prolapse and trapped fetal parts are unpredictable complications. Every practitioner knows this; and the literature, the courts, and the low baseline rate of such deliveries in North America highlight caution. Maternity units with interest and skill in delivering breech babies vaginally have achieved higher rates: 24% in the United States, 36% in Sweden, 38% in Israel, 38% in Switzerland, 39% in France, and 53% in Norway.^7-12 These retrospective studies carefully selected women for a trial of labour using various safety criteria and showed lower mortality and morbidity associated with vaginal breech delivery than in the term breech trial. Few obstetrical units other than Løvset's have published vaginal delivery rates as high as the term breech trial.¹²

Statistical power required a high vaginal delivery rate to enhance the trial's ability to detect small differences in outcome. With this aim, the researchers encouraged practitioners to increase their vaginal breech delivery rate beyond their previous comfort level. Despite being difficult to quantify, comfort level (or "practitioner comfort level") plays a pivotal part in the safety of complex procedures. Protected from medicolegal liability by the equipoise of a randomised trial, some practitioners must have pushed their comfort levels with vaginal breech delivery. This constitutes a significant bias: one of licence. The trial protocol's liberal labour guidelines allowed 0.5 cm dilation/h and 3.5 hours for the second stage. This is considerable licence, and few obstetricians from centres with proved safety in vaginal breech delivery would find them acceptable.

A discriminating procedure

Human skill varies. This is particularly evident when tasks are complex, require careful judgment, and have narrow margins for error. Increasingly complex tasks are discriminating, with more effort and skill required to master them. The safe vaginal delivery of a breech baby requires considerable skill and is a discriminating obstetrical procedure. Skill is required in multiple arenas: not just in the delivery technique, but in ultrasound assessment, the selection of cases, intrapartum fetal surveillance, the conduct of labour, and paediatric support. A coordinated, well functioning unit is difficult to quantify.

Other recognised discriminating procedures include carotid endarterectomy, where surgical skill is one of the strongest predictors of the operation's utility;^{13 14} surgery for cancer, where additional surgical training improves outcome;^{15 16} and vaginal hysterectomy. Rates of vaginal hysterectomy vary greatly among surgeons, and the learning curve to increase an individual surgeon's rate takes years.¹⁷ It has been suggested that "encouraging more surgeons to perform more vaginal hysterectomies may result initially in an increasing complication rate because it is technically more difficult."¹⁸

In the United Kingdom and in North America, the baseline vaginal hysterectomy rate for non-malignant disease is 20%. Encouraging a group of surgeons to suddenly increase their rate from 20% to 60% would not be a meaningful way to evaluate the safety of the procedure compared with abdominal hysterectomy. Nor should all women have an abdominal hysterectomy because some are poor candidates for vaginal surgery and because some surgeons lack the skill or experience to support a safe, high vaginal hysterectomy rate. Case selection depends on diagnosis, parity, size and mobility of the uterus, and operator skill, which together determine a safe baseline rate. Increasing the rate arbitrarily and randomising such a complex mix of patient and operator characteristics would compromise safety, yet this is what happened in the term breech trial.

Homogenising populations and clinicians

Randomisation improves the internal validity of trials by homogenising study and control populations thereby avoiding bias from differences between the two. Clinically important factors that are variable within populations are, however, homogenised as well. Their importance to the outcome is lost, and the trial loses external validity for individuals within subsets of the population. Results represent the mean outcome for all participants and are not applicable to subgroups at lower risk. Although subgroup analysis, found in the term breech trial, can potentially identify subpopulations in which a procedure may be safer, it is statistically weak. When phenomena are complex, many patient and practitioner characteristics influence outcome, rendering individual subgroups small and meaningful analysis of them difficult.

A major limit of evidence based medicine is the difficulty in applying the results of randomised trials to individual patients. For example, most would agree that a multiparous woman in advanced labour at 38 weeks with a 3000 g fetus in a frank breech presentation with flexed head and no nuchal cord represents a low risk subgroup of all breech presentations. By studying a heterogeneous group of women, the term breech trial lacks the external validity needed to guide us with the management of such women. Yet experienced obstetricians will now press for an emergency caesarean, not trusting their clinical judgment to discern low risk situations, because all women with a breech presentation have been assigned a similar risk status by a randomised controlled trial.

Multicentre trials can also lead to homogenisation of the intervention. Previous randomised trials of breech presentations were too small to detect clinically meaningful differences in outcome, so a large multicentre trial was required to improve statistical power. Yet despite the interest, altruism, and self referential experience of the practitioners, the involvement of 121 centres resulted in an average level of care. Encouraging practitioners to exceed their comfort level with vaginal breech delivery lowered that standard.

If generic levels of care had always been accepted as the ideal, none of the surgical subspecialties would have arisen. The standard of care shown by the term breech trial is not the best we can offer. Although breech deliveries commonly occur under average conditions, it does not mean that committed centres are unable to offer better than average care. Collectively we have been improving our "mean" level of care for years, and the perinatal risk associated with breech delivery has continued to drop despite stabilisation of the caesarean section rate.¹⁹

Simplified risk reduction

Historically, the greatest decrease in perinatal mortality from vaginal breech deliveries was reported by Bracht in 1938.²⁰ Other techniques recommended to enhance the safety of vaginal breech delivery include routine determination of fetal weight, head attitude, and nuchal or presenting cord using ultrasonography; continuous fetal monitoring; radiological pelvimetry; cautious attention to the progress of labour; and preparing for emergency symphysiotomy should fetal parts become trapped.²¹ Although poorly amenable to scientific analysis, some of these techniques are likely to be important for safe vaginal breech delivery. None were included in the term breech trial.

It is impractical for a large, multicentre trial to use complex risk reducing strategies. Meaningful quality control in 121 centres is impossible, and more caution would have meant fewer vaginal deliveries, increasing the number of participants needed to achieve similar statistical power. Therefore, the researchers chose a simple labour protocol with few risk avoidance strategies. The lack of proved effectiveness of other strategies ostensibly justified their exclusion; yet our current inability to analyse safe vaginal breech delivery does not preclude its existence. The resulting standard of care, arguably reasonable for a large, multicentre trial, falls short of its designation as the definitive study of vaginal breech delivery.

Since publication of the term breech trial, the onus has been placed on individual obstetrical units to retrospectively examine their experience with vaginal breech delivery and to show safety. Several have done so and continue to offer vaginal breech delivery.^{11 22 23} Safety in these specific centres is due to heterogeneity of human skill, not to statistical anomaly, and vaginal breech delivery in those units should be studied and emulated. For complex phenomena, a large, randomised, multicentre trial does not overrule demonstrated safety.

In the case of carotid endarterectomy, it should ideally be performed at a centre and by a surgeon with a perioperative stroke rate of 3%, not 6%. If unavailable, a patient might elect medical treatment, as the risks could outweigh the benefits. Similarly, a woman with an average breech presentation and access to average care may decide that a caesarean section is safer than a trial of labour; yet even that conclusion is potentially flawed: without a bias of licence, the maternity unit caring for her may well have a low, safe, baseline vaginal breech delivery rate.

Short term combined end points

Randomised trials often utilise short term end points because they are easier to measure than longer term outcomes. It is also easier to show a statistical difference in a combined end point rather than a single end point. Yet combined end points can be misleading.²⁴ In the term breech trial, the end point included perinatal mortality and various short term morbidities, including hypotonia, transient brachial plexus injury, and isolated low arterial cord pH or base excess, whose lasting significance is unclear. In countries with low perinatal mortality, this combined end point occurred in 5.7% of planned vaginal deliveries and 0.4% of women undergoing elective caesareans. Mortality was not significantly different (3 of 511 or 0.6%) in the planned vaginal delivery group compared with zero in the planned caesarean group. One of these deaths, included in the intention to treat analysis, occurred before the onset of labour in a cephalic twin weighing 1150 g, highlighting concerns about the adequacy of case selection and raising questions about the appropriateness of intention to treat analysis at all cost. Regardless, the impact of the trial's results was due primarily to an excess of short term morbidity in the planned vaginal delivery group.

Summary points When applied to complex phenomena, randomised trials have important limitations Vaginal breech delivery is a complex procedure that is poorly amenable to the methods of large multicentre randomised trials One randomised controlled trial has dictated a new standard of care for vaginal breech deliveries worldwide The use of a short term combined end point overstated any true risk of planned vaginal delivery to longer term neurodevelopmental outcome

Long term outcomes in breech babies are hard to assess epidemiologically, but were retrospectively shown to be equivalent in 1645 children, irrespective of the planned mode of delivery.²⁵ Researchers from the term breech trial have published details on death or abnormal neurodevelopment over two years in a subgroup of 923 children from the term breech trial.²⁶ They found a similar incidence: 2.8% in the planned vaginal delivery group and 3.1% in the elective caesarean section group. The use of a short term combined end point seems to have been misleading.

The limits of evidence based medicine

Delivering a breech presentation vaginally is a skill: guided by science, its safety relies on the experience of practitioners and caution. In the term breech trial, large scale randomisation, which homogenised both the study population and clinical intervention, resulted in an average level of care in an average population, limiting the trial's external validity in centres showing above average skill and in women of below average risk. Encouraging practitioners to exceed their comfort level ensured a high vaginal delivery rate and adequate statistical power, but introduced a bias of licence and compromised safety. Using a combined short term end point overstated any true effect on long term neurodevelopment.

The philosophical limits of evidence based medicine include failing to appreciate and cultivate the complex nature of sound clinical judgment, failing to appreciate the relevance of poorly quantifiable clinical phenomena that are obscured by randomisation, and devaluing the intangible differences between individuals, thus potentially devaluing them (patients and care providers).²⁷ The condemnation of vaginal breech delivery by one randomised controlled trial and its sweeping effect on clinical practice show how each of these philosophical limits can be exceeded.

---

The author thanks Ewart Woolley, whose skill in vaginal breech deliveries inspired him, and Robert Liston and Michael Klein for their support and direction.

Funding: None.

Competing interests: None declared.

Ethical approval: Not required.

References

1. Hannah ME, Hannah WJ, Hewson SA, Hodnett ED, Saigal S, Willan AR. Planned cesarean section versus planned vaginal birth for breech presentation at term: a randomised multicentre trial. Term Breech Trial Collaborative Group. Lancet 2000;356: 1375-83.[CrossRef][ISI][Medline]
2. Hauth JC, Cunningham FG. Vaginal breech delivery is still justified. Obstet Gynecol 2002;99: 1115-6.[Free Full Text]
3. Keirse MJ. Evidence-based childbirth only for breech babies? Birth 2002;29(1): 55-9.[CrossRef][ISI][Medline]
4. Halmesmaki E. Vaginal term breech delivery—a time for reappraisal? Acta Obstet Gynecol Scand 2001;80: 187-90.[CrossRef][ISI][Medline]
5. Lee KS, Khoshnood B, Sriram S, Hsieh HL, Singh J, Mittendorf R. Relationship of cesarean delivery to lower birth weight-specific neonatal mortality in singleton breech infants in the United States. Obstet Gynecol 1998;92: 769-74.[Abstract]
6. University of Toronto Maternal, Infant and Reproductive Health Research Unit. Term breech trial study protocol, 1996: 8.
7. Sanchez-Ramos L, Wells TL, Adair CD, Arcelin G, Kaunitz AM, Wells DS. Route of breech delivery and maternal and neonatal outcomes. Int J Gynecol Obstet 2001;73: 7-14.[CrossRef][Medline]
8. Lindqvist A, Lindeberg SN, Hanson U. Perinatal mortality and route of delivery in term breech presentations. Br J Obstet Gynaecol 1997;104: 1288-91.[ISI][Medline]
9. Schiff E, Friedman SA, Mashiach S, Hart O, Barkai G, Sibai BM. Maternal and neonatal outcome of 846 term singleton breech deliveries: seven-year experience at a single center. Am J Obstet Gynecol 1996;175: 18-23.[CrossRef][ISI][Medline]
10. Irion O, Almagbaly PH, Morabia A. Planned vaginal delivery versus elective cesarean section: a study of singleton term breech presentations. Br J Obstet Gynaecol 1998;105: 710-7.[ISI][Medline]
11. Kayem G, Goffinet F, Clement D, Hessabi M, Cabrol D. Breech presentation at term: morbidity and mortality according to the type of delivery at Port Royal Maternity hospital from 1993 through 1999. Eur J Obstet Gynecol Reprod Biol 2002;102: 137-42.[CrossRef][ISI][Medline]
12. Albrechtsen S, Rasmussen S, Reigstad H, Markestad T, Irgens LM, Dalaker K. Evaluation of a protocol for selecting fetuses in breech presentation for vaginal delivery or cesarean section. Am J Obstet Gynecol 1997;177: 586-92.[CrossRef][ISI][Medline]
13. Tu JV, Hannan EL, Anderson GM, Iron K, Wu K, Vranizan K, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med 1998;339: 1441-7.[Abstract/Free Full Text]
14. Barnett HJ, Taylor DW, Eliasziw M, Fox AJ, Ferguson GG, Haynes RB, et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med 1998;339: 1415-25.[Abstract/Free Full Text]
15. Mayer AR, Chambers SK, Graves E, Holm C, Tseng PC, Nelson BE, et al. Ovarian cancer staging: does it require a gynecologic oncologist. Gynecol Oncol 1992;47: 223-7.[CrossRef][ISI][Medline]
16. Tingulstad S, Skjeldestad F, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol 2003;102: 499-505.[Abstract/Free Full Text]
17. Varma R, Tahseen S, Lokugamage A, Kunde D. Vaginal route as the norm when planning hysterectomy for benign conditions: change in practice. Obstet Gynecol 2001;97: 613-6.[Abstract/Free Full Text]
18. Maresh M, Metcalfe M, McPherson K, Overton C, Hall V, Hargreaves J, et al. The VALUE national hysterectomy study: description of the patients and their surgery. Br J Obstet Gynaecol 2002;109: 302-12.
19. Albrechtsen S, Rasmussen S, Irgens LM. Secular trends in peri- and neonatal mortality in breech presentation; Norway 1967-1994. Acta Obstet Gynecol Scand 2000;79: 508-12.[CrossRef][ISI][Medline]
20. Bracht E. Zur Behandlung der Steisslage. Zentralblatt Gynaecol 1938;31: 1735-6.
21. Menticoglou SM. Symphysiotomy for the trapped aftercoming parts of the breech: a review of the literature and a plea for its use. Aust NZ J Obstet Gynaecol 1990;30: 1-9.[ISI][Medline]
22. Giuliani A, Schoell W, Basver A, Tamussino K. Mode of delivery and outcome of 699 term singleton breech deliveries at a single center. Am J Obstet Gynecol 2002;187: 1694-8.[CrossRef][ISI][Medline]
23. Alarab M, Regan C, O'Connell MP, Keane DP, O'Herlihy C, Foley ME. Singleton vaginal breech delivery at term: still a safe option. Obstet Gynecol 2004;103: 407-12.
24. Julian D. What is right and what is wrong about evidence-based medicine? J Cardiovasc Electrophysiol 2003:14(Suppl); S2-5.[CrossRef][ISI][Medline]
25. Danielian P, Wang J, Hall M. Long term outcome by method of delivery of fetuses in breech presentation at term: population based follow up. BMJ 1996;312: 1451-3.[Abstract/Free Full Text]
26. Whyte H, Hannah M, Saigal S. Outcomes of children at 2 years of age in the term breech trial. Am J Obstet Gynecol 2003;Suppl 189(6). [Abstract No 7.]
27. Tonelli MR. The philosophical limits of evidence-based medicine. Acad Med 1998;73: 1234-40.[ISI][Medline]

(Accepted 5 October 2004)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

Inappropriateness of randomised trials for complex phenomena: Single trial is never enough evidence to base decisions on

Isabelle Boutron, Philippe Ravaud, and Bruno Giraudeau
BMJ 2005 330: 94. [Extract] [Full Text]

Inappropriateness of randomised trials for complex phenomena: Are the results of the term breech trial generalisable?

Mary Hannah
BMJ 2005 330: 95. [Extract] [Full Text]

This article has been cited by other articles:

• Schmitz, T., Carnavalet, C. d. C., Azria, E., Lopez, E., Cabrol, D., Goffinet, F. (2008). Neonatal Outcomes of Twin Pregnancy According to the Planned Mode of Delivery. Obstet Gynecol 111: 695-703 [Abstract] [Full text]  
• Chinnock, M., Robson, S. (2007). Obstetric Trainees' Experience in Vaginal Breech Delivery: Implications for Future Practice. Obstet Gynecol 110: 900-903 [Abstract] [Full text]  
• Bellavite, P., Ortolani, R., Pontarollo, F., Piasere, V., Benato, G., Conforti, A. (2006). Immunology and Homeopathy. 4. Clinical Studies--Part 2. Evid Based Complement Alternat Med 3: 397-409 [Abstract] [Full text]  
• Hannah, M. (2005). Inappropriateness of randomised trials for complex phenomena: Are the results of the term breech trial generalisable?. BMJ 330: 95-95 [Full text]  
• Boutron, I., Ravaud, P., Giraudeau, B. (2005). Inappropriateness of randomised trials for complex phenomena: Single trial is never enough evidence to base decisions on. BMJ 330: 94-94 [Full text]  

Rapid Responses:

Read all Rapid Responses

Are the results of the Term Breech Trial generalisable?

Mary E Hannah, et al.
bmj.com, 5 Nov 2004 [Full text]

No Photographic Evidence

Yincent Tse
bmj.com, 9 Nov 2004 [Full text]

Recovering vaginal breech delivery

Andrew J Kotaska
bmj.com, 10 Nov 2004 [Full text]

Uncritical Acceptance of Clinical Trials

Harold Ewart Woolley
bmj.com, 15 Nov 2004 [Full text]

PLEASE DON'T THROW THE BABY OUT WITH THE BATHWATER

Isabelle Boutron, et al.
bmj.com, 17 Nov 2004 [Full text]

Randomized Trials Should Be Used for Complex Procedures

George D. Carson
bmj.com, 17 Nov 2004 [Full text]

Continuing vaginal breech birth: a German perspective

Michael Krause
bmj.com, 19 Nov 2004 [Full text]

External Validity and Internal Trial Conditions

Michael C Klein
bmj.com, 24 Nov 2004 [Full text]

Another reason why randomised controlled trials do not necessarily tell the whole truth in clinical medicine

Petra AMM De Sutter
bmj.com, 10 Jan 2005 [Full text]