Removing the hype from hypertensionBMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g1937 (Published 06 March 2014) Cite this as: BMJ 2014;348:g1937
- Matthew J Shun-Shin, academic clinical fellow in cardiology,
- James P Howard, academic clinical fellow in cardiology,
- Darrel P Francis, professor of cardiology
- 1International Centre for Circulatory Health, National Heart and Lung Institute, Imperial College London, London W2 1LA, UK
Earlier this year, Symplicity HTN-3, the world’s first blinded randomised controlled trial of renal denervation in hypertension, announced that it had failed to meet its targeted reduction in blood pressure.1 Before this announcement, three high profile publications—a case report2 and two unblinded trials3 4—had reported a consistent 30 mm Hg effect size, which was confirmed by more than two dozen reports of similar effect sizes. The intellectual property for the technology was sold for $800m (£480m; €582m).5 The various competing devices now number more than 50.
Symplicity HTN-3 faces scrutiny because the effect of renal denervation on blood pressure has fallen short of the target of about 10 mm Hg,6 which seemed modest given the results of the previous unblinded trials. However, unlike previous trials, Symplicity HTN-3 used a placebo controlled study design and blinded measurement of blood pressure. Therefore, the odd choice of study design was not that of Symplicity HTN-3, but rather the dozens of previous studies. Measurement of a noisy variable by unblinded optimistic staff is a known recipe for calamitous exaggeration.7 We should not expect the investigators of the blinded placebo controlled Symplicity HTN-3 trial to explain themselves. It is the hundreds of authors of the previous reports that need to speak up.
Why was the most reliable method for quantifying an effect size—namely, the randomised blinded placebo controlled trial—used only at this late stage? It is tempting to suggest that cost conscious early phase studies can jettison standard steps, such as randomisation and blinding, which are intended to minimise bias. But unblinded and non-randomised studies reliably deliver only unreliable answers.8
How can it be cheaper to get the wrong answer first and pay to repair it at a later date, rather than to get the right answer at the start? Two possibilities come to mind. Firstly, it might have been hoped that the correct answer would never come to light. But this would involve preventing blinded placebo controlled trials from starting, and somehow cancelling any that slipped through, which would be difficult. Secondly, an early exit might have been planned, leaving the cost (and consequences) to others.
Does my bias look big in this study?
Clinicians may misunderstand the term “bias” to represent malicious manipulation. In reality, bias is ubiquitous and varies only in size.7 Renal denervation is unusual only in its true effect size being so dramatically dwarfed by bias.9 The figure⇓ indicates powerful exaggerating forces that can be removed only by progressively reducing bias.
Peer review does not protect the literature against incorrect claims. An exaggeratedly positive study, once conducted, will probably be published somewhere and, if not, will still be unearthed and included in diligent meta-analyses.11 Two sorts of gatekeeper could prevent the conduct of trials without adequate protection against bias.
Firstly, ethics committees are responsible for protecting research participants’ “rights, safety, dignity, and wellbeing,”12 while balancing risks to participants against benefits to future patients. If the study is doomed to give the wrong answer, then no matter how small the risk, it is too large.
Renal denervation highlights the possibility that interventional trials aimed at measuring efficacy without randomisation or blinding might be worse than useless. If so, they could be unethical by default. They encourage patients to undergo experimental interventions, with the promise of providing information to help others, although they may unknowingly be contributing disinformation.
Ethics application forms have no section that focuses on actively reducing bias with steps such as randomising the intervention or blinding of the endpoint assessment. This must change.
The second potential gatekeepers are study participants, who should not be assumed to be passive. What if they were alert to the destructive capacity of bias arising from lack of randomisation or lack of blinding? What if they were empowered as the last line of defence for trials that slip past ethics committees? When approached to participate in a study that lacked these features without good reason, participants could report this omission to the local ethics committee. If the committee did not respond appropriately, the study participants could report the study and the local committee to a national committee. We could make this possible by including in children’s science education an understanding of how biased evaluation can distort results.
The real message from renal denervation may well be that trial design matters. To choose the wrong design is to choose the wrong answer.
Cite this as: BMJ 2014;348:g1937
Competing interests: We have read and understood the BMJ Group policy on declaration of interests and declare the following interests: We have conducted an uncontrolled and unblinded first-in-man safety study of renal denervation in heart failure (NCT01584700), which found a 7 mm Hg blood pressure drop. We immediately followed this with a blinded, placebo controlled randomised trial (NCT0163938). DPF is a consultant to Medtronic and Sorin.
Provenance and peer review: Not commissioned; externally peer reviewed.