Authors’ response: Critical appraisal of evidence on fracture prevention - daunting but essential
We thank our colleagues for insightful comments on our Too much medicine –analysis entitled ‘Overdiagnosis of bone fragility in the quest to prevent hip fracture’ 1. A number of correspondents understood the key message of our article that fractures in the elderly are caused mostly by falling - rather than osteoporosis, and that marginal evidence supports pharmacological prevention of hip fractures. These facts remain poorly recognised.
However, as some rapid responses included misunderstandings and untenable inferences, we welcome the opportunity to clarify some key issues here.
Many responses construe our Analysis as “focusing on hip fractures to the exclusion of other clinically important fractures”, while some argue that “fragility fractures cause excess mortality that can be remedied through bone-targeted pharmacotherapy”, “anti-fracture efficacy of bone-targeted pharmacotherapy is enhanced in secondary prevention and/or by treating those at high risk of fractures”, “moving from conventional BMD-based screening to the concept of fracture risk enhances screening/treatment efficacy”, and that “fracture liaison services are the new panacea for fracture epidemic”.
As intuitively rational as such assertions may seem, the arguments supporting them and the purported flaws in our paper do not stand up to scrutiny. We share our colleagues’ views that fractures are a serious health concern, and are aware that a multiple risk factor approach is commonly used in most non-communicable diseases. However, if such an approach has not been shown to reduce morbidity or mortality from serious fractures, reference to analogous practice in other fields of medicine is not rational. We reprimand children when they justify misbehaviours because their peers behaved similarly. Should we accept a lower ethical standard for our own work?
One repeated assertion was that our analysis focused on hip fractures to the exclusion of other clinically important fractures. The rationale is the importance of hip fractures to health, as stated in the opening paragraph of our paper: “hip fractures constitute a minority of fractures linked to osteoporosis, their consequences exceed those of all other fragility fractures combined.2 Vertebral fractures, recognised only by radiography, are of much less clinical concern (see appendix 1 on thebmj.com).3 4”.
A link to the “appendix 1”:
Vertebral fractures are often considered equivalent in importance to those of the hip. However, the diagnosis of vertebral fracture is notoriously arbitrary. Depending on criteria used to classify a change in an x-ray image as a fracture, the prevalence of vertebral fractures in an elderly population can vary from 3% to 90%. Referring to a “vertebral fracture epidemic” without clarifying the arbitrary nature of the definition of “fracture”, the fact that these are measures of the degree of vertebral compression rather than an actual break in the bone, and the asymptomatic nature of most “vertebral fractures” is analogous to raising alarm that the prevalence of prostate cancer in elderly men approaches 100%.
The arbitrary and inconsistent way that vertebral fractures have been defined affects the distinction between “primary” and “secondary” prevention trials of pharmacotherapy, as well as guidelines promoting a different treatment strategy for people with “existing/prevalent fractures”. Most of these prevalent fractures are deformations of the vertebrae, only visible on X-ray, with differing diagnostic thresholds affecting who is defined as a primary prevention or a secondary prevention patient.
Moreover, a symptomatic vertebral fracture is rarely “spontaneous” or purely osteoporotic; at least 50% are trauma-induced, particularly due to falling on the buttocks or lifting an object with straight knees 2-4. In a recent survey of vertebral fracture-related emergency department visits and hospitalizations in the older Dutch population, 83% of vertebral fractures were caused by a low-energy fall5. Seemingly benign activities such as bending or lifting light objects produce large loads on the spinal column (up to 10 times higher than with perfect posture) and are capable of fracturing a vertebra 4 6. Only one-third of the x-ray compressions, wrongly termed vertebral fractures, are symptomatic, and the occurrence of these vertebral compressions rather poorly predicts either the existence of back pain or functional status of the spine7.
Many rapid responses argue that “fragility fractures” cause excess mortality and that bone-targeted pharmacotherapy improves survival. While it is true that “fragility fractures” are associated with increased short and medium term mortality, this is also true for almost every illness of older adults. This association does not imply that the fractures caused the additional deaths; an elderly frail person is at higher risk both of fragility fractures, and of illness and death than a younger and less frail person. The relevant question is how much the increased morbidity and mortality linked to “fragility fractures” can be reduced by pharmacotherapy.
A recent study8 estimated the effect of hip fracture on excess mortality by following identical twin pairs discordant for hip fracture. In younger men and in women of any age, the excess risk of mortality lasted only one year after hip fracture. Only in men older than 75 did hip fracture affect mortality beyond the first year. This is a strong indication that previous studies overrated the excess mortality after a hip fracture in women, including the duration of the excess risk 9.
Rapid responses arguing for a mortality benefit of bone-targeted pharmacotherapy reference the “secondary prevention arm” of the HORIZON trial10 (HORIZON also had a “primary prevention arm” 11) but neglect its recognized flaws and questions about its results.12 In fact, immediately after publication of this “secondary prevention arm” of the HORIZON trial10, concern was expressed regarding the alleged mortality benefit (http://www.nejm.org/doi/full/10.1056/NEJMc073292). The authors responded (http://www.nejm.org/doi/pdf/10.1056/NEJMc073292) by stating that “the time-to-death analysis was a prespecified safety analysis that was described in both the protocol and the statistical-analysis plan (provided in the online Supplementary Appendix to our article)”. However, this statement might not be entirely accurate.
First, time to death was not a prespecified primary or secondary outcome nor a prespecified safety analysis as described in the primary study protocol (http://www.nejm.org/doi/suppl/10.1056/NEJMoa074941/suppl_file/nejmoa0749...) or in the updated protocol (http://www.nejm.org/doi/suppl/10.1056/NEJMoa074941/suppl_file/nejmoa0749...). The first mention we have found of a time to death analysis was in the statistical analysis plan (first release date June 5th 2006 and last revised March 28th, 2007). This was shortly before manuscript submission and after the HORIZON trial was completed.
Second, mortality was never listed in the Clinicaltrials.gov database as any kind of outcome. Of most concern, according to the history of changes to the HORIZON trial registration (https://clinicaltrials.gov/archive/NCT00046254/2006_05_31/changes), even the primary outcomes were added only on May 31, 2006, that is, four months after patient recruitment was completed.
Third, a total of 242 deaths were included in the analysis of the HORIZON secondary prevention trial 10 and the analysis rendered a hazard ratio of 0.72 (95% CI 0.56-0.93) for death in those allocated to zoledronic acid. While subsequent fractures were associated with death, they explained only 8% of the zoledronic acid effect on mortality13 12. In the HORIZON primary prevention trial11, an identical number of deaths occurred (n=242) but the relative risk of death in the zoledronic acid treatment arm was 1.16 (95% CI 0.91-1.49). The HORIZON trialists claimed that the difference in estimates was due to limited study power in the primary prevention trial14 but given the similar number of outcomes this is not a likely explanation. A further puzzling finding is the protective effect against fatal arrhythmias apparent in the “secondary prevention arm” of the HORIZON trial 10, whereas in the “primary prevention arm” of the HORIZON trial 11, the incidence of arrhythmias was paradoxically increased among the zoledronic acid treated group.
The proposed mechanism for zoledronic acid to reduce the risk of lethal infectious and cardiovascular diseases is by modifying the pro-inflammatory and immunologic effects of hip fracture 12. However, elevated serum markers of inflammation return to normal levels within the first month after the fracture in most patients.13 15 16 Infusion of zoledronic acid within the first month after the fracture did not reduce mortality rates 14, although this was the period with postulated highest efficacy. On the contrary, the statistically significant difference was observed with infusion more than 12 weeks after the fracture 14. This seems incompatible with the proposed pathogenetic mechanism for the mortality reduction benefit 12. Interestingly, the trial was initially designed to include patients within a week after hip fracture, but was later modified to include patients for up to 12 weeks. Mortality survival curves started to diverge only after 16 months of follow-up (median follow-up 23 months).10 12
Another possible explanation is differences in loss to follow-up. While incomplete follow-up was a minor problem in the primary prevention trial11, in the secondary prevention trial, 155 participants withdrew consent or were lost to follow-up (out of a total of 295 subjects who did not complete follow-up) in the zoledronate arm and 137 in the placebo arm (out of 316 who did not complete follow-up)10. These numbers correspond to a relative risk of consent withdrawal or loss to follow-up in zoledronic acid users of 1.21 (95% CI 1.03-1-43; p=0.02) compared with those on placebo. Consent withdrawal or loss to follow-up are likely to be related to higher rate of disability and death.17-19 The trial was also stopped early, following an additional unplanned interim analysis. Early termination of trials has been shown to overestimate reported benefits, with a ratio of relative risks versus non-truncated trials of 0.71 (95% CI 0.65-0.77), regardless of whether statistical stopping rules are used20. This bias is thought to be due to random fluctuations that occur during the conduct of a trial.
Deviation from intention to treat analyses inflates treatment effect estimates21, and this seems to hold true here, too: after exclusion of the “secondary prevention arm” of the HORIZON trial10 from a meta-analysis9, the alleged mortality benefit of osteoporosis therapy vanished (RR 0.94 with 95% CI 0.84-1.06; p=0.31).
Until other randomized controlled trials designed a priori to assess mortality replicate the results of the secondary prevention HORIZON trial, we recommend caution in interpreting these results.
Regarding the general anti-fracture efficacy of bone-targeted pharmacotherapy, many responses referred to drug-specific Cochrane systematic reviews published in 2008 22-24, but none disputed the findings of our more recent and comprehensive systematic review (Figure 3).
Dr. Treadwell highlights the fundamental difference between primary and secondary prevention. As he notes, some correspondents infer that secondary prevention could represent a more productive opportunity for pharmacotherapy. As intuitively obvious as this seems, what truly is the evidence? Our systematic review attempted to discriminate trials of primary prevention (enrolment based on low BMD alone) from secondary prevention (prior vertebral fractures only). This proved very challenging, notably because of the variable definitions of “vertebral fractures”. Additionally, our exploratory subgroup analysis of trial results for populations that have been defined as primary versus secondary (based on the presence of previous vertebral fractures) failed to reveal a significant interaction effect. This finding not only supports the decision to report the results of all trials as a single estimate only, but also questions the conventional wisdom that drugs are more potent in secondary prevention.
The only “true” secondary prevention trial is the “secondary prevention arm” of HORIZON10, for which patients were randomized to intravenous zoledronic acid or placebo after having sustained a hip fracture. We have addressed above some concerns about the execution of the trial, including early termination, outcome reporting, and loss to follow-up, but it is also worth reviewing the effectiveness evidence. With a median follow-up of nearly two years, hip fracture rates did not differ significantly between zoledronic acid and placebo: 2.0% on zoledronic acid vs. 3.5% on placebo, NS. There were fewer clinical non-vertebral fractures, 7.6% vs. 10.7% (an absolute risk reduction (ARR) of 3.1% and and a relative risk reduction (RRR) of 27%, p=0.03), and fewer clinical vertebral fracture, 1.7% in the zoledronic acid group vs. 3.8% in the placebo group (ARR 2.1% and RRR 55%, p=0.02). Effects of pharmacotherapy remained modest even in this secondary prevention population. The great majority of patients do not benefit.
Many authors question the validity of providing a single NNT, 175 over three years of bisphosphonate treatment to prevent one hip fracture,, as the estimate is only for one fracture type, albeit the most devastating. We similarly restricted our estimate of the potential harms of treatment to a single, most devastating one – atypical fracture of the femur (NNH of 300 for a three-year treatment period25 26). Accordingly, we feel that a trade-off of 175 to gain vs. 300 to be harmed is a relatively fair comparison, except that the former comes from “unrealistic” efficacy studies whereas the latter is derived from ordinary healthcare settings (real life).
Several rapid responses also note that the field is moving from conventional BMD-based screening to the concept of “fracture risk” (identification of those at highest risk of fractures). As already briefly discussed above, this is another intuitively rational strategy that lacks proper evidence.
Dr. Chika E Uzoigwe notes possible biases in studies that failed to show anti-fracture efficacy in people with higher risk of fracture, but does not provide any evidence that pharmacotherapy is effective in this population.
It has become common for doctors to feel driven to deliver any and all available treatments to their patients, even when there is little hope of a benefit and real potential for harm. Many rapid responses postulated the existence of a treatment or care gap affecting individuals who have suffered a fracture but are left “untreated”. Naturally we agree that society should seek to prevent fractures in the elderly. Those who have already suffered a fracture represent a logical target for intervention. However noble the intentions, they do not eliminate the responsibility to abstain from interventions that have not been proven to be effective, for which there is a poor ratio of potential benefit versus potential harm, or whose cost precludes more effective alternative strategies.
Several responses also suggest using various non-BMD-based approaches to decide whom to treat. Although such approaches are tempting, there is still no reliable evidence that they are effective.
Many correspondents argued that fracture liaison services (FLS), programs “to identify and treat osteoporosis in high risk individuals” are the panacea for a future fracture epidemic. However, the scientific basis/rationale rests either on low-quality (observational, non-RCT) evidence or modeling-type cost-effectiveness extrapolations, the validity of which we have questioned previously27. As appealing as such programs appear, their effectiveness can only be proven by appropriately designed comparative-effectiveness RCTs which take account of overall costs as well as clinical outcomes.
To be treated or not: who should make the decision? Advocates of more aggressive intervention do not seem to endorse shared-decision making28. Whilst many of the organisations criticizing our paper advocate a 3% 10-year risk of hip fracture (or 20% risk of ‘major osteoporotic fracture’) as being high enough to justify drug therapy, patients consider the appropriate threshold of 10-year risk to be 50% (sic!)29. Patients may not – when fully informed – want pharmacological fracture prevention.
Competing interest: BMJ asks authors of all contributions in The BMJ (including rapid responses) to declare their competing interests. Some authors of rapid responses with obvious conflicts have chosen not to do so. We share BMJ’s stance on competing interest: “A competing interest—often called a conflict of interest—exists when professional judgment concerning a primary interest (such as patients' welfare or the validity of research) may be influenced by a secondary interest (such as financial gain or personal rivalry). It may arise for the authors of an article in The BMJ when they have a financial interest that may influence, probably without their knowing, their interpretation of their results or those of others.”
We are convinced that open and unemotive discussion of the current scientific evidence on how best to reduce the fracture burden can benefit patients in the long run. Orthodoxy, inattention to methodological issues in RCTs, and lack of transparency concerning competing interests are barriers to achieving the best and most cost-effective results.30-33
1. Jarvinen TL, Michaelsson K, Jokihaara J, et al. Overdiagnosis of bone fragility in the quest to prevent hip fracture. Bmj 2015;350:h2088.
2. Cooper C, Atkinson EJ, Kotowicz M, et al. Secular trends in the incidence of postmenopausal vertebral fractures. Calcified tissue international 1992;51(2):100-4.
3. Cooper C, Atkinson EJ, O'Fallon WM, et al. Incidence of clinically diagnosed vertebral fractures: a population-based study in Rochester, Minnesota, 1985-1989. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research 1992;7(2):221-7.
4. Myers ER, Wilson SE. Biomechanics of osteoporosis and vertebral fracture. Spine (Phila Pa 1976) 1997;22(24 Suppl):25S-31S.
5. Oudshoorn C, Hartholt KA, Zillikens MC, et al. Emergency department visits due to vertebral fractures in the Netherlands, 1986-2008: steep increase in the oldest old, strong association with falls. Injury 2012;43(4):458-61.
6. Duan Y, Seeman E, Turner CH. The biomechanical basis of vertebral body fragility in men and women. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research 2001;16(12):2276-83.
7. O'Neill TW, Cockerill W, Matthis C, et al. Back pain, disability, and radiographic vertebral fracture in European women: a prospective study. Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA 2004;15(9):760-5.
8. Michaelsson K, Nordstrom P, Nordstrom A, et al. Impact of hip fracture on mortality: a cohort study in hip fracture discordant identical twins. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research 2014;29(2):424-31.
9. Haentjens P, Magaziner J, Colon-Emeric CS, et al. Meta-analysis: excess mortality after hip fracture among older women and men. Annals of internal medicine 2010;152(6):380-90.
10. Lyles KW, Colon-Emeric CS, Magaziner JS, et al. Zoledronic acid and clinical fractures and mortality after hip fracture. The New England journal of medicine 2007;357(18):1799-809.
11. Black DM, Delmas PD, Eastell R, et al. Once-yearly zoledronic acid for treatment of postmenopausal osteoporosis. The New England journal of medicine 2007;356(18):1809-22.
12. Colon-Emeric CS, Mesenbrink P, Lyles KW, et al. Potential mediators of the mortality reduction with zoledronic acid after hip fracture. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research 2010;25(1):91-7.
13. Sedlar M, Kudrnova Z, Erhart D, et al. Older age and type of surgery predict the early inflammatory response to hip trauma mediated by interleukin-6 (IL-6). Archives of gerontology and geriatrics 2010;51(1):e1-6.
14. Eriksen EF, Lyles KW, Colon-Emeric CS, et al. Antifracture efficacy and reduction of mortality in relation to timing of the first dose of zoledronic acid after hip fracture. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research 2009;24(7):1308-13.
15. Beloosesky Y, Hendel D, Weiss A, et al. Cytokines and C-reactive protein production in hip-fracture-operated elderly patients. The journals of gerontology Series A, Biological sciences and medical sciences 2007;62(4):420-6.
16. Baehl S, Garneau H, Le Page A, et al. Altered neutrophil functions in elderly patients during a 6-month follow-up period after a hip fracture. Experimental gerontology 2015;65:58-68.
17. May GS, DeMets DL, Friedman LM, et al. The randomized clinical trial: bias in analysis. Circulation 1981;64(4):669-73.
18. Juni P, Altman DG, Egger M. Systematic reviews in health care: Assessing the quality of controlled clinical trials. Bmj 2001;323(7303):42-6.
19. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Bmj 2010;340:c869.
20. Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA : the journal of the American Medical Association 2010;303(12):1180-7.
21. Abraha I, Cherubini A, Cozzolino F, et al. Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study. Bmj 2015;350:h2445.
22. Wells G, Cranney A, Peterson J, et al. Risedronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane database of systematic reviews 2008(1):CD004523.
23. Wells GA, Cranney A, Peterson J, et al. Alendronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane database of systematic reviews 2008(1):CD001155.
24. Wells GA, Cranney A, Peterson J, et al. Etidronate for the primary and secondary prevention of osteoporotic fractures in postmenopausal women. Cochrane database of systematic reviews 2008(1):CD003376.
25. Schilcher J, Koeppen V, Aspenberg P, et al. Risk of atypical femoral fracture during and after bisphosphonate use. N Engl J Med 2014;371(10):974-6.
26. Schilcher J, Koeppen V, Aspenberg P, et al. Risk of atypical femoral fracture during and after bisphosphonate use. Acta Orthop 2015;86(1):100-7.
27. Jarvinen TL, Sievanen H, Kannus P, et al. The true cost of pharmacological disease prevention. Bmj 2011;342:d2175.
28. Stiggelbout AM, Van der Weijden T, De Wit MP, et al. Shared decision making: really putting patients at the centre of healthcare. Bmj 2012;344:e256.
29. Douglas F, Petrie KJ, Cundy T, et al. Differing perceptions of intervention thresholds for fracture risk: a survey of patients and doctors. Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA 2012;23(8):2135-40.
30. Jarvinen TL, Jokihaara J, Guy P, et al. Conflicts at the heart of the FRAX tool. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne 2013.
31. Collins GS, Michaelsson K. Fracture risk assessment: state of the art, methodologically unsound, or poorly reported? Current osteoporosis reports 2012;10(3):199-207.
32. Bolland MJ, Grey A, Gamble G, et al. Comment on Kanis et al.: Pitfalls in the external validation of FRAX. Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA 2013;24(1):389-90.
33. Collins GS, Mallett S, Altman DG. Predicting risk of osteoporotic and hip fracture in the United Kingdom: prospective independent and external validation of QFractureScores. Bmj 2011;342:d3651.
Competing interests: TLNJ is he Jane and Aatos Erkko foundation clinical professor of Orthopedics and Traumatology at the University of Helsinki and is supported by unrestricted academic grants from the Academy of Finland and the Sigrid Juselius Foundation.