Effectiveness of vertebroplasty using individual patient data from two randomised placebo controlled trials: meta-analysis
BMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d3952 (Published 12 July 2011) Cite this as: BMJ 2011;343:d3952
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
The comparisons of generic health status is in this article based on
EQ-5D, a standardised instrument for measuring health often used in
economical evaluations. This measure is, however, statistically
problematic, and failing to find an advantage of vertebroplasty over
placebo by testing mean values, is not necessarily an indication of
vertebroplasty not having an effect.
This may seem paradoxical, but it depends on EQ-5D being constructed
to have a bi- or tri-modal distribution (depending on whether subjects in
full health are included or not). The individual pairwise differences from
randomisation to follow up will therefore have a non-Gaussian multi-modal
distribution with even further modes.
The mean value from such a distribution is, in contrast to the mean
value from a Gaussian distribution, not representative for the entire
population of studied patients. A comparison of mean values says therefore
very little about actual treatment effects. Major treatment effects could
be present among some patients without this affecting the mean value. A
more sophisticated analysis of the subpopulations represented by the
different modes would have to be performed.
Competing interests: No competing interests
Addendum:
As noted by Dr. van Schalkwyk, blinding represented an essential aspect of
the US-based trial and he correctly points out that the results must be
interpreted in light of blinding efficacy. For some reason he failed to
discuss our published detailed analysis of blinding (1). This showed that
if any unblinding occurred it was limited to the control group. Such
unblinding likely would have diminished the observed benefit in the
control group. The fact that the benefit was similar between the
treatment and control groups, even in the setting of potential unblinding
in the control group, only strengthens the conclusion that the efficacy of
vertebroplasty may have been overstated in uncontrolled trials.
Brinjikji W, Comstock BA, Heagerty PJ, et al. Investigational
Vertebroplasty Efficacy and Safety Trial: detailed analysis of blinding
efficacy. Radiology 2010(1):219-25.
Competing interests: No competing interests
The point raised by Dr van Schalkwyk has no bearing on the result of
our meta-analysis as we only included data up to one month. There was
essentially no crossover before the 1-month primary efficacy endpoint in
the US-based trial.1
While Dr van Schalkwyk is correct in stating that many more control
patients than patients who underwent vertebroplasty crossed over,
crossover was not a study outcome but was used to enhance patient
enrollment. As we have stated previously2, differential crossover does not
affect the conclusion of no differences between groups at 1 month. However
what Dr van Schalkwyk fails to note was that the individual patient data
meta-analysis found a statistically significantly higher proportion of
patients who underwent vertebroplasty taking opioid analgesics at 1 month
compared to those in the control group (RR 1.25 (95% CI 1.1 to 1.4)3, a
finding that does not support his arguments.
We have previously postulated that in the trial by Kallmes et al.,
patients who continued to have bothersome pain after intervention were
given opioid analgesics if they had been randomly assigned to
vertebroplasty but were offered crossover if they had been randomly
assigned to the control intervention.2 This is further supported by the
fact that the rate of opioid withdrawal over time did not differ between
groups in the Australian trial.4
The results of our meta-analysis of combined individual patient level
data from two blinded trials of vertebroplasty are entirely valid and do
not demonstrate any benefit of vertebroplasty over a sham control, even
for selected subgroups.
1. Kallmes DF, Comstock BA, Heagerty PJ, Turner JA, Wilson DJ,
Diamond TH, et al. A randomized trial of vertebroplasty for osteoporotic
spinal fractures. N Engl J Med 2009;361:569-79.
2. Kallmes DF, Jarvik JG, Osborne RH, Comstock BA, Staples MP,
Heagarty PJ, et al. Clinical utility of vertebroplasty: Elevating the
evidence. Radiology 2010:255(3):675-80.
3. Staples MP, Kallmes DF, Comstock BA, Jarvik JG, Osborne RH,
Heagarty PJ, et al. Effectiveness of vertebroplasty using individual
patient data from two randomised placebo controlled trials: meta-analysis.
BMJ 2011;342:d3952.
4. Buchbinder R, Osborne RH, Ebeling PR, Wark JD, Mitchell P, Wriedt
C, et al. A randomized trial of vertebroplasty for painful osteoporotic
vertebral fractures. N Engl J Med 2009;361:557-68.
Competing interests: RB is supported in part by an Australian National Health and Medical Research Council practitioner fellowship and DFK has received research support from Stryker and ArthroCare and is a consultant for CareFusion.
Dear Editors
The recent meta-analysis by Staples et al [BMJ 2011; 343:d3952]
concluded that vertebroplasty failed to show advantage over placebo. The
quality of a meta-analysis however must depend on the quality of the
original studies.
True value in a study often resides in the unexpected. This is
certainly the case in one of the two studies on which the meta-analysis is
based. In the study of Kallmes et al, [N Engl J Med 2009; 361:569-579] a
remarkable finding was the discrepancy between patients who crossed over
from the 'sham' group to the active treatment group, and the reverse. The
proportions were respectively 27/63 and 8/68. This finding needs to be
explained, as the likelihood of this being a mere statistical fluctuation
is under 1:10,000 (Fisher's 'exact' test, two-tailed, using R version
2.13.1).
There are three logical possibilities here:
1. The intervention was substantially more effective than placebo, but the
study otherwise failed to identify this success, presumably related to
methodological flaws;
2. Blinding was broken, and those in the placebo group consequently
perceived substantially less benefit from the sham procedure than from the
real procedure;
3. Blinding was broken and the intervention was nevertheless more
effective.
We conclude that, in any case, the study was flawed. It will likely
be a matter of opinion whether such a substantial difference can be
accounted for solely by impaired blinding, but other analyses within the
study that examined patients' ability to distinguish between sham and real
treatment do not support possibility #2. This in turn suggests that the
intervention was more effective than placebo.
How might the study methodology be flawed? As most other aspects of
the design of the study seem quite reasonable, the obvious candidates are
use of verbal reported scores in the past 24 hours, and the Roland-Morris
Disability Questionnaire. (We note that in the study, the cutoff criterion
for the latter was post-specified). It would seem logical to impute that
use of these and similar measures should be regarded with suspicion in
this context.
Cross-over with the prospect of further spinal probing may be a more
robust measure of failure of a procedure (sham or real) than numerical
scoring systems.
We conclude that meta-analysis of dross must produce dross, a
platitude that seems to have been ignored here and perhaps in other meta-
analyses that are then used to alter clinicians' behaviour.
Disclaimer: I have no vested interest in vertebroplasty, and very
rarely endorse use of this intervention in management of back pain.
Competing interests: No competing interests
From the classical descriptions in literature considering surgical
descriptors such as 'Postoperative spine' (Mall JC et al 1983) to
recent descriptions in autologous/heterologous grafting, a long-road
has been followed up in this area.
The typical lateral or posterolateral grafts between transverse and
articular apophyses to intercorporal joints bridging intervertebral disc
space may be illustrative.
The vertebral osseus alterations, including vertebral and post-surgical back syndrome and results in thin section CT with reconstitution
algorithms, may be compared with MRI results in defining soft tissue, thecal area, spinal cord and spinal nerves.
Conceptual role of dural integrity in disc (vertebral) surgery and, obviously, in traumatic surgery is the hallmark, considering the
valvular mechanism-onset risk in surgical methodics (inasmuch as
fistulous risk is less obvious in this type of condition).
The role of anterior procedure in fracture with corporectomy, repositioning with grafts or acrylic resins and/or osteosynthesis, have been
classical techniques in this standpoint.
Ref.
-Epstein NE. Treatment of fractures of the vertebral limbus and spinal
stenosis in five adolescents and five adults. Neurosurg 1984 ; 24 : 595-604.
-Wagner A. Diagnostic imaging in fracture of the lumbar vertebral ring apophyses. Acta Radiol 1992 ; 33: 72.
-Lopez Escobar M. Neurosurgery. H Puerta del Mar 2000
Competing interests: No competing interests
The Australasian Musculoskeletal Medicine Association members were
very critical of the proposed vertebroplasy in 2005 when it was first
presented to them. We have all been proved correct. Dorsal ramus
injections, above and below the crushed vertebra of LA and Steroid work
extremely well, are repeatable and do not carry all the problems.
1. In Vivo Temperature Profile of Intervertebral Discs and Vertebral Endplates During Vertebroplasty: An Experimental Study in Sheep
Aebli, Nikolaus, Goss, Benjamin G., Thorpe, Paul, Williams, Richard P., & Krebs, Jorg (2006) In Vivo Temperature Profile of Intervertebral Discs and Vertebral Endplates During Vertebroplasty: An Experimental Study in Sheep. Spine: an international journal for the study of the spine, 31(15), pp. 1674-1678.
Official URL: http://journals.lww.com/spinejournal/Abstract/2006...
2. Prediction of adjacent vertebral fracture risk associated with vertebroplasty : a computer-based simulation study
Langton, C. M. & Ahwal, F. (2009) Prediction of adjacent vertebral fracture risk associated with vertebroplasty : a computer-based simulation study. Bone, 44(1), S84-S84.
DOI: 10.1016/j.bone.2009.01.187
Official URL: http://www.sciencedirect.com/science?_ob=Publicati...
Competing interests: No competing interests
RCTs on Vertebroplasty: The Underlying Data is the Limitation to the Usefulness of a Meta-Analysis
The fundamental problems with the 2 RCTs that underwent a joint
subgroup meta-analysis here have been discussed in a number of
publications.
Since the meta-analysis of a given data set cannot overcome shortcomings
in the generation of that data, the following points need to be taken into
account, when evaluating the authors' conclusions from their meta-
analysis.
Buchbinder et al. 1 were able to recruit only 78 out of 468 screened
patients (16.6%) over a period of more than 4 years.
These patients were treated in 4 different centers, 3 of which performed
an average of only 1.1 to 3.3 procedures (verum and placebo procedures)
per year.
Kallmes and collaborators 2 were able to include only 131 out of 1813
screened patients (7.2%) over a period of also more than 4 years.
The patients in this study were treated in 11 different centers (5 USA, 5
UK, 1 Australia) and with the exception for the leading institution (Mayo
Clinic, n=30, 22.9%) and the Australian study center (n=22, 16.8%), no
exact numbers are given for the individual centers.
It is stated, however, that 27 patients (20.6%) were treated in the
remaining 4 US centers and 52 patents (39.7%) in the 5 UK centers.
This calculates to an average procedure frequency of between 1.7 and 3.3
per year (verum and placebo procedures).
Given that the patient numbers in both studies are not very large, the
long recruitment periods, the number of participating centers and the low
frequency of procedures in some of the centers cast some doubt on the
study design.
In clinical experience, patients with fresh osteoporotic vertebral
fractures (OVFx) benefit most from vertebroplasty, as has been shown in a
number of papers and rather recently by Rousing et al. as well as by
Farrokhi et al. 3 4.
Results after 3 months, however, don't seem to differ that much, which to
a considerable part is due to the improvement in patients with OVFx under
conservative therapy.
It is a very strange finding on this background, that neither Kallmes nor
Buchbinder found a significant or even clinically relevant pain
improvement in their placebo groups.
This represents a fundamental difference to the findings by Wardlaw et al.
in their RCT on kyphoplasty 5 and by Rousing et al. in their randomized
trial on vertebroplasty 4 and leads to the suspicion that the patients
included by Kallmes and Buchbinder were fundamentally different from those
in the trials by other investigators.
Neither Kallmes nor Buchbinder required the location of maximum pain
on clinical examination to match the fracture location on imaging, which
would be a standard tool in routine clinical decision making.
In fact, clinical examination as a determinator is not mentioned in either
manuscript.
Considering that the one-year prevalence of low back pain for older adults
in the literature ranges between 22 and 65 % 6, there might have been a
considerable number of patients, whose pain was not or no longer directly
related to their OVFx, which ties in well with the lack of improvement in
the control groups.
Beyond that, is not uncommon for older persons with poor sagittal spinal
balance and reduced muscular stabilization to start suffering from severe
myofascial or sacroiliac joint pain after an OVFx.
These individuals can be identified by careful clinical examination and
obviously are not likely to benefit much from a vertebral augmentation
procedure.
In our experience, the great majority of patients with painful, fresh
OVFx will chose to undergo vertebroplasty when offered.
Buchbinder et al. state in their paper, that 30% of potential candidates
refused to participate.
However, when doing the math on the numbers in both papers, 64% of
qualifying patients refused to participate (as did 70% in the Kallmes
trial).
Since the (perceived?) beneficial effect of vertebroplasty had already
become established public knowledge by the time of recruitment, it is
quite possible that the most pain-ridden patients opted to receive
vertebroplasty outside the trials rather than running the risk of
receiving placebo treatment.
This raises the clear possibility of a bias due to 'selective subject
participation' 7, potentially rendering generalization from the sample to
the entire population of such patients invalid.
The participating patients might have been systematically different from
the non-participants, regarding baseline characteristics, clinical
characteristics and prognosis.
Since OVFx tend to be most painful when fresh 4 5, this bias might in part
also explain the strikingly high proportion of older fractures recruited
into both trials.
In summary, both, the Kallmes and the Buchbinder groups have
performed formidable work.
Unfortunately, they have both largely failed to study the kinds of
patients that we would normally treat with vertebroplasty.
Their recent subgroup meta-analysis changes nothing about that.
So are their findings reason enough to stop performing vertebroplasty for
OVFx?.
We believe not.
References:
1. Buchbinder R, Osborne RH, Ebeling PR, Wark JD, Mitchell P, Wriedt C, et
al. A randomized trial of vertebroplasty for painful osteoporotic
vertebral fractures. N Engl J Med 2009;361(6):557-68.
2. Kallmes DF, Comstock BA, Heagerty PJ, Turner JA, Wilson DJ, Diamond TH,
et al. A randomized trial of vertebroplasty for osteoporotic spinal
fractures. N Engl J Med 2009;361(6):569-79.
3. Farrokhi MR, Alibai E, Maghami Z. Randomized controlled trial of
percutaneous vertebroplasty versus optimal medical management for the
relief of pain and disability in acute osteoporotic vertebral compression
fractures. Journal of neurosurgery. Spine 2011;14(5):561-9.
4. Rousing R, Andersen MO, Jespersen SM, Thomsen K, Lauritsen J.
Percutaneous vertebroplasty compared to conservative treatment in patients
with painful acute or subacute osteoporotic vertebral fractures: three-
months follow-up in a clinical randomized study. Spine (Phila Pa 1976)
2009;34(13):1349-54.
5. Wardlaw D, Cummings SR, Van Meirhaeghe J, Bastian L, Tillman JB,
Ranstam J, et al. Efficacy and safety of balloon kyphoplasty compared with
non-surgical care for vertebral compression fracture (FREE): a randomised
controlled trial. Lancet 2009;373(9668):1016-24.
6. Walker BF. The prevalence of low back pain: a systematic review of the
literature from 1966 to 1998. J Spinal Disord 2000;13(3):205-17.
7. Kramer MS, Shapiro SH. Scientific challenges in the application of
randomized trials. JAMA 1984;252(19):2739-45.
Competing interests: No competing interests