CCBYNC Open access
Research

# Low intensity pulsed ultrasound for bone healing: systematic review of randomized controlled trials

BMJ 2017; 356 (Published 22 February 2017) Cite this as: BMJ 2017;356:j656
1. Stefan Schandelmaier, methodologist1 2,
2. Alka Kaushal, physician1 3,
3. Lyubov Lytvyn, methodologist4,
4. Diane Heels-Ansdell, biostatistician1,
5. Reed A C Siemieniuk, methodologist1 5,
6. Thomas Agoritsas, assistant professor1 6,
7. Gordon H Guyatt, distinguished professor1 7,
8. Per O Vandvik, associate professor8 9,
9. Rachel Couban, medical librarian3,
10. Brent Mollon, orthopedic surgeon10,
11. Jason W Busse, associate professor1 3 11
1. 1Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main St West, Hamilton, ON L8S 4L8, Canada
2. 2Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Spitalstrasse 12, CH-4031 Basel, Switzerland
3. 3Michael G. DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, ON L8S 4K1, Canada
4. 4Oslo University Hospital, Forskningsveien 2b, Postboks 1089, Blindern 0317 Oslo, Norway
5. 5Department of Medicine, University of Toronto, 200 Elizabeth Street, Toronto, ON, M5G 2C4, Canada
6. 6Division General Internal Medicine and Division of Clinical Epidemiology, University Hospitals of Geneva, Rue Gabrielle-Perret-Gentil 4, CH-1211, Geneva, Switzerland
7. 7Department of Medicine, McMaster University, 1280 Main St West, Hamilton, ON L8S 4L8, Canada
8. 8Institute of Health and Society, Faculty of Medicine, University of Oslo, 0318 Oslo, Norway
9. 9Department of Medicine, Innlandet Hospital Trust-division, Gjøvik, Norway
10. 10Orillia Soldiers’ Memorial Hospital, 41 Frederick Street, Orillia, ON L3V 5W6, Canada
11. 11Department of Anesthesia, McMaster University, Hamilton, ON L8S 4K1, Canada
1. Correspondence to: S Schandelmaier, Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main St W, Hamilton, ON L8S 4L8, Canada schandes{at}mcmaster.ca
• Accepted 1 February 2017

## Abstract

Objective To determine the efficacy of low intensity pulsed ultrasound (LIPUS) for healing of fracture or osteotomy.

Design Systematic review and meta-analysis.

Data sources Medline, Embase, CINAHL, Cochrane Central Register of Controlled Trials, and trial registries up to November 2016.

Study selection Randomized controlled trials of LIPUS compared with sham device or no device in patients with any kind of fracture or osteotomy.

Review methods Two independent reviewers identified studies, extracted data, and assessed risk of bias. A parallel guideline committee (BMJ Rapid Recommendation) provided input on the design and interpretation of the systematic review, including selection of outcomes important to patients. The GRADE system was used to assess the quality of evidence.

Results 26 randomized controlled trials with a median sample size of 30 (range 8-501) were included. The most trustworthy evidence came from four trials at low risk of bias that included patients with tibia or clavicle fractures. Compared with control, LIPUS did not reduce time to return to work (percentage difference: 2.7% later with LIPUS, 95% confidence interval 7.7% earlier to 14.3% later; moderate certainty) or the number of subsequent operations (risk ratio 0.80, 95% confidence interval 0.55 to 1.16; moderate certainty). For pain, days to weight bearing, and radiographic healing, effects varied substantially among studies. For all three outcomes, trials at low risk of bias failed to show a benefit with LIPUS, while trials at high risk of bias suggested a benefit (interaction P<0.001). When only trials at low risk of bias trials were considered, LIPUS did not reduce days to weight bearing (4.8% later, 4.0% earlier to 14.4% later; high certainty), pain at four to six weeks (mean difference on 0-100 visual analogue scale: 0.93 lower, 2.51 lower to 0.64 higher; high certainty), and days to radiographic healing (1.7% earlier, 11.2% earlier to 8.8% later; moderate certainty).

Conclusions Based on moderate to high quality evidence from studies in patients with fresh fracture, LIPUS does not improve outcomes important to patients and probably has no effect on radiographic bone healing. The applicability to other types of fracture or osteotomy is open to debate.

Systematic review registration PROSPERO CRD42016050965

## Introduction

For over 20 years, patients have used low intensity pulsed ultrasound (LIPUS) as an adjunct treatment to improve bone healing. Based on radiographic outcomes, the US Food and Drug Administration and the UK National Institute for Health and Care Excellence (NICE) have approved LIPUS for fracture healing.1 2 Depending on country and device, LIPUS devices currently cost around £1000-4000 (€1170-4700; $1250-5000). In 2008, 45% of trauma surgeons in Canada prescribed bone stimulators to manage tibia fractures, equally split between LIPUS and electrical stimulation (21% each).3 In 2006 sales from LIPUS amounted to about$250m in the US alone.3 4

Within the past seven years, 10 systematic reviews have assessed the effectiveness of LIPUS for bone healing.5 6 7 8 9 10 11 12 13 14 Because existing randomized controlled trials were limited by small sample size, risk of bias, inconsistent results, and failure to cover outcomes important to patients, no review offered definitive conclusions. All reviews identified the need for additional randomized controlled trials. In addition, recent reviews used suboptimal strategies for outcome selection, data synthesis, analysis, and interpretation, leading to potentially misleading conclusions. For instance, the most recent systematic review,11 published in the top specialty journal for orthopedic surgeons, considered radiographic union a “critically important outcome” and did not assess the effect of LIPUS on the outcomes of pain relief or re-operation, which are important to patients. Their conclusion that “LIPUS treatment effectively reduces the time to radiographic fracture union” is questionable because it is based on the pooled absolute difference in days to healing, which does not account for the large variation in healing time, showed high unexplained heterogeneity (I2=94%), and was driven by studies at high risk of bias. This positive conclusion has the potential to expand the already considerable use of a potentially ineffective treatment.

This systematic review is part of the BMJ Rapid Recommendations project, a collaborative effort from the MAGIC research and innovation program (www.magicproject.org) and BMJ. The aim of the project is to respond to new potentially practice changing evidence and provide a trustworthy practice guideline in a timely manner.15 In this case, the publication of the TRUST trial,16 a multicentre trial that randomized 501 patients with tibia fractures and has cast doubt on the effectiveness of LIPUS, initiated the process. This systematic review informed a parallel guideline published in a multi-layered electronic format on bmj.com17 and MAGICapp (https://www.magicapp.org/public/guideline/mL6yYj).

We assessed whether LIPUS, compared with sham device or no device, improved outcomes important to patients and radiographic healing in those with any kind of fracture or osteotomy.

## Methods

### Guideline panel and patient involvement

According to the BMJ Rapid Recommendations process,15 a guideline panel provided critical oversight to the review and identified populations, subgroups, and outcomes of interest. The panel included six content experts (five orthopedic or trauma surgeons and one physiotherapist), six methodologists (four of whom are also front line clinicians), and four patients with personal experience of fractures (one of whom had used LIPUS). All patients received personal training and support to optimize contributions throughout the guideline development process. The members of the patient panel led the interpretation of the results based on what they expected the typical values and preferences of patients to be, as well as the variation between patients.

### Information sources

We searched Medline, PubMed, Embase, CINAHL, and the Cochrane Central Register of Controlled Trials up to 16 November 2016, using a combination of keywords and MeSH terms for fracture, orthopedic surgical procedures, and ultrasound. Additional searches included trials registries clinicaltrials.gov and isrctn.com. An experienced research librarian designed the search strategies (appendix 1). Two independent reviewers scanned the references from eligible studies, related systematic reviews, and all studies citing eligible randomized controlled trials on Google Scholar.

### Study selection

We included randomized controlled trials that compared LIPUS with a sham device or no device in patients with any type of fracture regardless of location (long bone or other bone), type (fresh fracture, delayed union, non-union, or stress fracture), or clinical management (operative or non-operative). We included any type of osteotomy, including distraction osteogenesis. We excluded trials published only as protocols or abstracts if we were unable to get the final results from investigators.

Two reviewers, independently and in duplicate, screened the titles and abstracts of identified articles and acquired the full text of any article that either reviewer judged to be potentially eligible. They independently applied the eligibility criteria to the full texts and, when consensus could not be reached, resolved disagreements through discussion or adjudication by a third reviewer.

### Data collection

Two reviewers used standardized forms to independently abstract data; they resolved disagreements by discussion or involved a third reviewer when required. Extracted data included characteristics of patients and fractures, clinical management, risk of bias, intervention details, statements about compliance with treatment, and outcomes.

### Risk of bias assessment

Two reviewers independently assessed risk of bias using a modified Cochrane risk of bias instrument that includes response options of “definitely or probably yes” (assigned a low risk of bias) or “definitely or probably no” (assigned a high risk of bias), an approach we have previously validated.18 On the study level, we assessed generation of randomization sequence, concealment of allocation, blinding of patients, caregivers, and outcome reporting (by comparing each publication with its corresponding published protocol, when available). For each outcome within studies, we assessed blinding of outcome assessors, loss to follow-up, and additional limitations. We considered ≥20% loss to follow-up to represent a high risk of bias, unless the investigators performed appropriate sensitivity analyses showing the robustness of the results. We categorized a trial as being at low risk of bias for a particular outcome if we identified no limitation for any risk of bias item. As a post hoc sensitivity analysis, we alternatively considered a more conservative threshold of ≥10% loss to follow-up because the categorization of three trials depended on this criterion (19%,16 28%,19 and 31%20 loss to follow-up for the outcome radiographic healing).

### Outcomes

Patients identified functional recovery (time to return to work and time to full weight bearing), pain reduction, and number of operations for subsequent fracture or related to osteotomy (re-operation for operatively managed fracture and osteotomy) as the most important outcomes for patients considering LIPUS for bone healing. Because many clinicians currently base their management on time to radiographic healing, a surrogate outcome important only insofar as it influences patient experience, the panel requested its inclusion in our review. We extracted all outcomes that fell into these categories as well as adverse effects related to the ultrasound device.

### Synthesis of results

We pooled treatment effects of LIPUS on similar outcomes across eligible trials, regardless of clinical subgroups, focusing on complete case analysis. We calculated pooled estimates and associated 95% confidence intervals using random effects models for meta-analysis with three or more studies and fixed effects models for meta-analysis with two studies. We examined heterogeneity associated with all pooled analyses using both the χ2 test and I2 statistic. SAS version 9.4, R version 3.1, and Review Manager 5.3 provided software for the statistical analysis.

For time to event outcomes, we pooled hazard ratios. For studies that did not apply methods of survival analysis, we considered time to event reported as a continuous variable (for example, days to return to work) at the longest follow-up time. We used the relative effect measure ratio of means (mean LIPUS/mean control) to account for the baseline difference in fracture healing depending on type of bone and (such as scaphoid, clavicle, tibia, etc) and fracture or procedure (such as stress fracture or distraction osteogenesis). We pooled the natural logarithm of the ratio of means and presented the results as percentage difference (relative change). For studies that reported the proportion of patients who achieved the event at a specific time point, we calculated risk ratios.

When studies used different instruments to measure the same construct on a continuous scale, we converted all instruments to the most commonly used instrument among studies and then pooled results using the weighted mean difference.21

For the outcomes number of subsequent operations and adverse events related to the device, we calculated both risk ratios, which are preferable in case of varying baseline risks, and risk differences, which allow inclusion of studies with zero events in both groups.

In consultation with the expert and patient guideline panel, we prespecified three subgroup hypotheses to explain heterogeneity of effects between studies: LIPUS will show larger effects in studies at high risk of bias; effects will differ based on clinical subgroups; and LIPUS will show larger in studies effects with greater patient compliance. In consultation with the six clinical experts on the parallel guideline panel, we classified eligible randomized controlled trials according to the following five clinical subgroups: operatively managed fresh fractures, non-operatively managed fresh fractures, stress fractures, non-union, and osteotomy (including distraction osteogenesis). Because compliance was inconsistently reported, two reviewers independently categorized trials using response options of “definitely or probably high compliance” or “definitely or probably moderate compliance,” using as a guide a definition of high compliance as at least 80% of patients who applied LIPUS for at least 80% of the total time prescribed. We conducted univariable tests of interaction to establish if the effect size from the subgroups differed significantly from each other and, to test independence of subgroup effects, performed multivariable meta-regression in which we included risk of bias (high versus low), compliance with LIPUS treatment (high versus moderate), and clinical subgroups (as above) as independent variables in a single model.

Only one outcome, days to radiographic healing, included enough studies to perform all planned subgroup analysis. We had prespecified in our protocol at least three studies per group. We assessed the credibility of significant subgroup effects using the criteria suggested by Sun and colleagues.22 Based on the finding that risk of bias seemed to independently explain the high heterogeneity in the outcome days to radiographic healing, we performed subgroup analysis by risk of bias for all outcomes.

The authors and the guideline panel achieved consensus in categorizing the quality of evidence for all reported outcomes as high, moderate, low, or very low using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. In the GRADE approach, randomized controlled trials begin as high quality but can be rated down because of risk of bias, inconsistency, indirectness, imprecision, or publication bias.23 We considered rating down for inconsistency if the magnitude and direction of effects were dissimilar, the confidence intervals had minimal overlap, the test of heterogeneity was significant, or the I2 was high.24 For outcomes with 10 or more studies, we inspected symmetry of funnel plots and performed Egger’s statistical test for publication bias.25

To calculate absolute effects, we applied the effect estimate from the meta-analysis to the control arm of the TRUST trial, which enrolled patients with tibia fractures and had the largest sample size of any eligible study that was at low risk of bias. The approach to rating certainty of individual outcomes was fully contextualized—that is, in rating quality about any individual outcome, we took into account the findings on the other outcomes.

### Patient involvement

Four patient representatives were full members of the guideline and contributed to the selection and prioritization of outcomes, values and preferences assessments, and critical feedback to the protocol for the systematic review and the BMJ Rapid Recommendations manuscript.

## Results

### Search results

We identified 3849 potentially eligible abstracts, retrieved 42 studies in full text, and found 26 eligible randomized controlled trials (fig 1).16 19 20 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 Two trials, Handolin and colleagues32 33 and Emami and colleagues,29 30 provided two publications reporting on the same group of patients. There were no shared patients between the TRUST pilot26 and the definitive trial.16 Our registry search yielded four protocols of potentially eligible trials; one was discontinued because of slow recruitment (ISRCTN90844675, personal communication, outcome data not available yet), one manuscript is under peer review (NCT00744861, personal communication: “no difference between the control group and the ultrasound group”), one is completed but unpublished (JPRN-UMIN000002005, no response from investigators), and the last is still ongoing (NCT02383160). Attempts to acquire the full text of another potentially eligible study,51 reported in a recent systematic review,11 were unsuccessful.

Fig 1 Studies included in review of low intensity pulsed ultrasound compared with control (sham device or no device) for patients with fracture or osteotomy

### Study characteristics

Eligible trials enrolled patients with operatively managed fresh fractures (n=7); non-operatively managed fresh fractures (n=6); stress fractures (n=2); non-unions (n=3); and osteotomies (n=8), of which five were distraction osteogenesis (table 1). Most trials enrolled patients with tibia fractures or osteotomies (n=14). All but two trials applied LIPUS for 20 minutes every day, either for a fixed period or until radiographic healing. Otherwise, one trial applied LIPUS for 15 minutes a day,36 and another trial for five minutes every second day.39 Fifteen trials (60%) provided their control group with an inactive device that was indistinguishable from the active LIPUS. Only three trials (12%) were explicitly free from industry funding.28 42 48

Table 1

Characteristics of studies included in review of low intensity pulsed ultrasound for bone healing after fracture

View this table:

### Risk of bias

We contacted authors to resolve areas of uncertainty and successfully clarified details in five randomized controlled trials.32 33 35 37 40 We considered six trials to be at low risk of bias,16 26 29 37 46 47 and the remaining 20 studies to be at high risk of bias (table 2). The main limitations were failure to report method for allocation concealment (15 trials), unblinded patients (10 trials), unblinded caregivers or outcome assessors (10 trials), and high or unclear numbers of patients excluded from the analysis (13 trials; table 2).

Table 2

Risk of bias in studies included in review of low intensity pulsed ultrasound for bone healing after fracture

View this table:

### Outcomes

Table 3 summarizes findings of all outcomes. Interactive tables summarizing findings are available online at https://www.magicapp.org/public/guideline/mL6yYj.

Table 3

GRADE summary of findings in review of low intensity pulsed ultrasound for bone healing after fracture

View this table:

#### Functional recovery

Only the TRUST trial assessed time to return to work with a time to event analysis and found no significant effect (hazard ratio 1.11 favoring control, 95% confidence interval 0.82 to 1.50; 343 patients).16 Three trials assessed the number of days to return to work; the pooled effect was not significant (2.7% later return with LIPUS, 95% confidence interval 7.7% earlier to 14.3% later; I2=0%; 392 patients) (fig 2). We found no significant interaction with risk of bias (P=0.86). If we consider an alternative threshold of ≥10% loss to follow-up for assessing risk of attrition bias, all three studies would fall into the category of high risk of bias. Given the consist absence of effects, however, this would not lower our confidence in the result. A fourth trial in patients with delayed union of tibia fracture provided insufficient data for inclusion in meta-analysis (table 2) but reported no significant difference in days to return to work.50

Fig 2 Difference in days to return to work after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device)

Only the TRUST trial assessed time to full weight bearing with a time to event analysis and found no significant effect (hazard ratio 0.87 favoring LIPUS, 95% confidence interval 0.70 to 1.08; 451 patients). Three trials assessed the number of days to full weight bearing. Overall results suggested no significant effect on full weight bearing with LIPUS but high heterogeneity (I2=95%). The effect of the one trial at high risk of bias (40.0% earlier, 48.4% to 30.3 earlier)35 differed significantly from the consistent results from the two trials at low risk of bias (4.8% later, 4.0% earlier to 14.4% later; 483 patients; interaction P<0.001, subgroup effect not effected by alternative threshold for missing data) (fig 3).

Fig 3 Difference of days to full weight bearing after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device), by risk of bias. Interaction P<0.001

Appendix 2 presents results of other functional outcomes including return to leisure activities, return to household activities, return to level of function before injury, and physical function measured with a multidimensional questionnaire. None of these was significantly affected by use of LIPUS, nor did they show substantial inconsistency.

#### Pain reduction

Four trials assessed pain, two with a 100 mm visual analogue scale37 49 and two with the “bodily pain” subdomain of the SF-36 instrument.16 26 After we transformed all results to a 100 mm visual analogue scale, findings at three to six weeks’ follow-up showed no significant effect of LIPUS on pain reduction but high heterogeneity (I2=97%). The effect of the one trial at high risk of bias (28.12 mm lower, 95% confidence interval 37.05 to 19.19 lower)39 differed significantly from the consistent results from the three trials at low risk of bias (0.93 mm lower, 2.51 lower to 0.64 higher; 626 patients; I2=0%; interaction P<0.001; fig 4). The subgroup effect was no longer significant when we used a threshold of ≥10% missing data to designate a trial at high risk of attrition bias (P=0.35, fig A in appendix 3). Two other small studies assessed pain intensity at five months but could not be included in the meta-analysis. One reported pain outcomes only narratively (no effect),41 and another used a modified instrument with unclear scale and variance (no effect).49

Fig 4 Mean difference of pain reduction after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device) by risk of bias. All instruments transformed to 0-100 visual analogue scale. Interaction P<0.001

Other outcomes for pain included pain intensity assessed at multiple time points and number of painful days (appendix 4). None showed a significant effect of LIPUS nor substantial inconsistency.

#### Number of subsequent operations

Ten trials reported the number of subsequent operations, including three trials reporting zero events in both arms. Neither the pooled risk ratio (0.80 in favor of LIPUS, 95% confidence interval 0.55 to 1.16; I2=0%; seven trials, 693 patients; fig 5) nor the pooled risk difference (3% reduction with LIPUS, 7% reduction to 2% increase; I2=0%; 10 trials, 740 patients) showed a significant effect. There was no significant interaction with risk of bias on either scale (risk ratio: P=0.75; risk difference: P=0.64). The results did not depend on the threshold for missing data.

Fig 5 Risk ratio of number of subsequent operations related to fracture after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device)

#### Time to radiographic healing

Two trials used time to event analysis methods to assess time to radiographic healing16 26 and showed no significant effect of LIPUS (hazard ratio 1.06 in favor of control, 95% confidence interval 0.86 to 1.32; I2=0%; 532 patients). Fifteen trials reported the number of days to radiographic healing. Overall results suggested accelerated radiographic healing with LIPUS (26% earlier, 33.6% to 17.8% earlier; I2=85%). The effect differed significantly between the 12 trials at high risk of bias (32.8% earlier, 39.5% to 25.3% days earlier; I2=78%; 446 patients) and the three trials at low risk of bias (1.7% earlier, 11.2% earlier to 8.8% later, I2=10%; 483 patients; interaction P<0.001; fig 6). This subgroup effect fulfilled eight of nine credibility criteria relevant to risk of bias as an explanation of heterogeneity (table 4). In addition, the subgroup effect was robust to our sensitivity analysis using a more conservative threshold for defining risk of attrition bias (interaction P=0.004, fig B in appendix 3). The effect of LIPUS on days to radiographic healing did not differ significantly across clinical subgroups (interaction P=0.13, fig C in appendix 3) or between high and moderate compliance with treatment (interaction P=0.99, fig D in appendix 3). In our multivariable meta-regression, which included risk of bias, clinical subgroups, and compliance with treatment, the only significant effect modifier was the risk of bias (P=0.005).

Fig 6 Percentage difference in days to radiographic healing after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device), by risk of bias. Interaction P<0.001

Table 4

Credibility of subgroup effects for risk of bias for outcome days to radiographic healing in studies of low intensity pulsed ultrasound for bone healing

View this table:

Another randomized controlled trial in patients with delayed union of tibia fracture reported only the proportion of healed fractures at 16 weeks and did not find a significant difference (65% in the LIPUS and 46% in the control arm, P=0.07; high risk of bias towards LIPUS because of serious imbalance in age of fracture at baseline).44

The funnel plot based on time to radiographic healing was not clearly asymmetrical, and Egger’s test for publication bias was not significant (P=0.25, fig E in appendix 3).

#### Adverse effects related to device

Seven studies explicitly reported the absence of any adverse effects related to the device; two other studies reported mild transient skin irritations in six patients. The pooled risk ratio based on these two studies (2.65 in favor of control, 95% confidence interval 0.32 to 22.21; 129 patients) was not significant, nor was the pooled risk difference based on all nine trials (0%, 1% reduction to 1% increase; I2=0%; 839 patients; fig 7). We found no significant interaction with risk of bias on the risk difference scale (P=0.75).

Fig 7 Risk difference in adverse effects related to ultrasound device after fracture treated with low intensity pulsed ultrasound (LIPUS) compared with control (sham device or no device)

## Discussion

### Main findings

Our systematic review provides moderate quality evidence that LIPUS applied to patients with fractures or osteotomies has no effect on time to return to work or the number of subsequent operations (table 3). Overall results suggested a possible reduction of days to full weight bearing, pain, and radiographic healing, but with large variability between studies strongly associated with risk of bias as an effect modifier: only trials with high risk of bias showed benefit. Based on randomized controlled trials at low risk of bias, we found high quality evidence that LIPUS has no effect on pain reduction, days to full weight bearing, or adverse effects related to the device, and moderate quality evidence that LIPUS has no effect on days to radiographic healing (table 3).

### Comparison with other systematic reviews

Our results are consistent with other systematic reviews in concluding that most randomized controlled trials on LIPUS treatment are poorly reported, lack outcomes important to patients, and are at high risk of bias.5 6 7 8 9 10 11 12 13 14 Our systematic review, however, differs from previous systematic reviews in several important aspects. Firstly, we include the recently published TRUST trial,16 by far the largest trial on LIPUS treatment for bone healing, which reported a number of outcomes important to patients. Secondly, our choice of outcomes and interpretation of findings were informed by a guideline panel including patients with personal experience of fractures in the context of BMJ Rapid Recommendations. Patients considered functional recovery, pain reduction, and operations as critical outcomes, while expressing little interest in the commonly reported surrogate outcome of radiographic healing. Thirdly, we used optimal statistical approaches and, in particular, the ratio of means to combine days to radiographic healing, return to work, or full weight bearing across studies. This relative effect measure is most appropriate in the context of LIPUS, where the average time to recovery differs substantially between clinical subgroups. For instance, a lower grade stress fracture is likely to heal much faster than a complicated tibia fracture. It is not surprising, therefore, that previous meta-analyses found high heterogeneity when they used absolute mean differences to pool across studies.8 11 12

Finally, we used the GRADE approach to assess the quality of evidence, taking into account the results of subgroup analysis based on risk of bias: when effects differed significantly between high and low quality trials, we based our conclusions on trials at low risk of bias. Our approach of limiting conclusions to trials at low risk of bias depends on our judgment of risk; however, our ratings of risk of bias were consistent with those of a previous Cochrane systematic review.5 Furthermore, most trials judged to be at high risk of bias had limitations in more than one domain, and some had additional sources of bias including baseline imbalance or unclear clustering when patients had more than one fracture or operation. Application of our risk of bias judgments as an effect modifier met eight of nine relevant criteria for a credible subgroup analysis (table 4). A post hoc sensitivity analysis exploring a more conservative threshold for attrition bias (≥10% loss to follow-up) yielded, for all outcomes, results essentially consistent with the primary analyses

### Limitations

The primary limitation of our review is the failure of most trials to measure or report outcomes important to patients. Of the 26 eligible trials, 11 reported, in sufficient detail for inclusion in meta-analysis, outcomes that patients consider critical for decision making.16 26 27 29 31 32 33 35 37 39 46 47 Of these, the only four trials that contributed substantial data included either patients with operatively managed fresh tibia fracture16 26 29 or conservatively managed clavicle fracture.37 One could question the extent to which our results apply to patients not included at all (such as children) or under-represented (stress fractures, non-union, and osteotomies) in the eligible trials. Qualitative subgroup effects (such as no benefit in one subgroup and important benefit in another) are, however, unusual. In the absence of evidence to the contrary, it might therefore be reasonable to apply our results to these populations. Our subgroup analysis and meta-regression for radiographic healing found no effect modification based on clinical subgroups. Certainly, the burden of proof regarding the effect of LIPUS in children and under-represented populations rests with those who might postulate a benefit.

### LIPUS compared with electrical stimulation

Our findings are similar to those of a 2016 systematic review of 15 small trials that explored electrical stimulation versus sham treatment for fracture healing, only four of which were at low risk of bias.52 This review found moderate quality evidence for a 35% reduction (95% confidence interval 19% to 47%; I2=46%) in the rate of radiographic non-union. The authors found no evidence of a subgroup effect based on clinical presentation (that is, fresh fractures, delayed union or non-union, spinal fusion, or surgical osteotomy; interaction P=0.41). They did not explore whether risk of bias explained heterogeneity, but all four trials at low risk of bias showed no significant effect on radiographic union.53 54 55 56 They found a small reduction in pain (mean difference of −7.7 mm on a 100 mm visual analogue scale for pain, −13.92 to −1.43) and low quality evidence for no difference in functional outcome (mean difference of −0.88 points on 100 point SF-36 physical component summary score, −6.63 to 4.87).

### Conclusions

In conclusion, moderate to high quality evidence shows that LIPUS fails to accelerate return to work, return to full weight bearing, and pain reduction or reduce the need for subsequent operation. If one gives highest credibility to combined effects from all available randomized controlled trials, low quality evidence would suggest a large reduction in time to radiographic healing. If, however, one gives higher credence to trials at low risk of bias, moderate to high quality evidence suggests that LIPUS not only has no effect on outcomes important to patients but also fails to accelerate radiographic healing. The evidence applies directly to patients with fresh fractures. The applicability to children and other under-represented populations, particularly those with non-union, for which no trustworthy direct evidence exists, is open to debate.

#### Linked articles in this BMJ Rapid Recommendations cluster

• • Poolman RW, Agoritsas T, Siemieniuk RAC, et al. Low intensity pulsed ultrasound (LIPUS) for bone healing: a clinical practice guideline. BMJ 2017;356:j576. doi:10.1136/bmj.j576

• summary of the results from the Rapid Recommendation process

• Magic App (www.magicapp.org)

• expanded version of the results with multilayered recommendations, evidence summaries, and decision aids for use on all devices

#### What is already known on this topic

• Low intensity pulsed ultrasound (LIPUS) devices are marketed worldwide to accelerate recovery from a fracture or osteotomy

• Previous systematic reviews provided no definite conclusions about the effect of LIPUS on outcomes important to patients and radiographic healing

#### What this study adds

• A guideline panel including patients and clinical experts informed outcome selection, importance of outcomes, subgroup analyses, and interpretation of results

• Subgroup analyses suggested that beneficial effects of LIPUS are restricted to trials at high risk of bias

• With inclusion of the recently published TRUST trial, sufficient high quality data for patients with fresh fractures has accumulated to conclude that LIPUS fails to improve outcomes important to patients and radiographic healing

## Footnotes

• We thank members of the Rapid Recommendations panel for critical feedback on outcome and subgroup selection, GRADE judgments, and manuscript feedback, including Rudolf Poolman (chair and orthopedic surgeon), Ian Harris (orthopedic and trauma surgeon), Inger Schipper (trauma surgeon), Maureen Smith (patient representative), Alexandra Albin (patient representative), Sally Nador (patient representative), William Sasges (patient representative), Ton Kuijpers (methodologist), Loes van Beers (physiotherapist), and Michael Verhofstad (trauma surgeon).

• Contributors: JWB, RACS, GHG, and POV conceived the study idea. SS and JWB coordinated the systematic review. SS wrote the first draft of the manuscript. RC designed the search strategy. LL, AK, RC, and SS screened abstracts and full texts. LL, AK, RACS, TA, and SS acquired the data and judged risk of bias in the studies. SS, JWB, and DHA performed the data analysis. DHA and GHG provided statistical advice. SS, RACS, BM, POV, JWB, and GHG interpreted the data analysis. All authors critically revised the manuscript. SS had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. SS is guarantor.

• Funding: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

• Competing interest: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare that JWB, DHA, and GHG were co-authors of the TRUST trial, which was supported in part by an industry grant from Smith & Nephew, a manufacturer of LIPUS devices.

• Ethical approval: Not required.

• Data sharing: No additional data available.

• Transparency: The lead author (the manuscript’s guarantor) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Abstract