The relative clinical effectiveness of ranibizumab and bevacizumab in diabetic macular oedema: an indirect comparison in a systematic reviewBMJ 2012; 345 doi: http://dx.doi.org/10.1136/bmj.e5182 (Published 13 August 2012) Cite this as: BMJ 2012;345:e5182
- John A Ford, research assistant1,
- Andrew Elders, statistician1,
- Deepson Shyangdan, research fellow2,
- Pamela Royle, senior research fellow2,
- Norman Waugh, professor of public health medicine and health technology assessment2
- 1Health Services Research Unit, University of Aberdeen, Health Services Building, Aberdeen AB25 2ZD, UK
- 2Warwick Evidence, Division of Health Sciences, Warwick Medical School, Coventry, UK
- Correspondence to: J A Ford
- Accepted 16 July 2012
Objective To indirectly compare the effectiveness of ranibizumab and bevacizumab in the treatment of diabetic macular oedema.
Design Systematic review and indirect comparison.
Data sources Medline (1996–September 2011), Embase (1996–September 2011), and the Cochrane Central Register of Controlled Trials (Issue 4, 2011).
Selection criteria for studies Randomised trials evaluating ranibizumab or bevacizumab in diabetic macular oedema with a common comparator and sufficient methodological similarity to be included within an indirect comparison were eligible for inclusion.
Main outcome measures The primary outcome was the proportion of patients with an improvement in best corrected visual acuity of more than two lines on the Early Treatment Diabetic Retinopathy Study (ETDRS) scale. Secondary outcomes included mean changes in best corrected visual acuity and in central macular thickness, and adverse events. Best corrected visual acuity was converted to logMAR units, a linear scale of visual acuity with positive values representing increasing visual loss. Indirect comparisons were done using Bayesian methods to estimate relative treatment effects of bevacizumab and ranibizumab.
Results Five randomised controlled trials with follow-up of 6–12 months and a common comparator (multiple laser treatment) were sufficiently similar to be included in the indirect comparison. Generally studies were small, resulting in wide credible intervals. The proportions of patients with an improvement in best corrected visual acuity of >2 lines were 21/77 participants (27%) for bevacizumab and 60/152 participants (39%) for ranibizumab (odds ratio 0.95 (95% credible interval 0.23 to 4.32)). The wide credible intervals cannot exclude a greater improvement, or worse outcome, for either drug. The mean change in best corrected visual acuity non-significantly favoured bevacizumab (treatment effect −0.08 logMAR units (−0.19 to 0.04)). The difference in mean change in central macular thickness was not statistically significant between ranibizumab and bevacizumab (treatment effect −6.9 μm (−88.5 to 65.4)).
Conclusions Results suggest no difference in effectiveness between bevacizumab and ranibizumab, but the wide credible intervals cannot exclude the possibility that either drug might be superior. Sufficiently powered, direct head to head trials are needed.
Diabetic retinopathy is a major cause of visual loss and a leading cause of blindness.1 Diabetic macular oedema, a common complication of diabetic retinopathy, is caused by accumulation of excess extracellular fluid in the macula, disruption of the blood-retina barrier, and abnormal permeability, associated with increased levels of vascular endothelial growth factor.2 Visual impairment caused by macular oedema may be reversible in the early stages, but prolonged oedema causes irreversible damage.
Laser photocoagulation has been the mainstay of treatment for diabetic macular oedema and is soundly evidence based.3 However, laser treatment mainly preserves vision rather than restoring it, and some patients do not respond.4 New treatments targeting vascular endothelial growth factor, such as ranibizumab, bevacizumab, and pegaptanib, have shown promise not only in preserving vision but also in improving it.5 6 7 8 These treatments have been widely used in age related macular degeneration, and their use in diabetic macular oedema is growing.
Bevacizumab (Avastin, Genentech/Roche) targets all isoforms of vascular endothelial growth factor and was developed to restrict vascular growth in the treatment of colorectal and other cancers. It has been widely used outside its licensed indication9 as an intravitreal treatment for macular oedema. Ranibizumab (Lucentis, Genentech/Roche) is a fragment of the same parent molecule as bevacizumab. It is considerably more expensive than bevacizumab (costs of £50–£105 for bevacizumab v £742 for ranibizumab). The cost of ranibizumab to the UK National Health Service is lower because of a patient access scheme that is currently confidential. In a previous study of classic age related macular degeneration, it was estimated that ranibizumab would have to be 40% more effective at preventing visual loss than bevacizumab to justify the marginal costs and achieve no more than £30 000 per quality adjusted life year.10 Pegaptanib (Macugen, Eyetech Pharmaceuticals/Pfizer) is a PEGylated aptamer, with a high affinity to vascular endothelial growth factor 165 and was licensed for the treatment of exudative age related macular degeneration in 2004 but was not approved by the National Institute of Health and Clinical Excellence (NICE).11
There have been no trials directly comparing ranibizumab and bevacizumab for treatment of diabetic macular oedema, although one is under way in Austria.12 NICE has recently carried out a technology appraisal of ranibizumab for diabetic macular oedema. Ranibizumab, even with the patient access scheme, was not found to be cost effective compared with laser photocoagulation.13 The decision was appealed by Novartis, Royal College of Nursing, Royal College of Ophthalmologists, and jointly from four patient groups. NICE rejected these appeals in November 2011.14
Our aim was to compare the clinical effectiveness of ranibizumab and bevacizumab as measured by best corrected visual acuity and central macular thickness in diabetic macular oedema.
A literature search was performed to identify randomised controlled trials evaluating bevacizumab or ranibizumab in treating diabetic macular oedema. We searched Medline (1996–September 2011), Embase (1996–September 2011), and the Cochrane Central Register of Controlled Trials (issue 4, 2011). There were no language restrictions. The flow of studies is shown in fig 1⇓.
The search terms for Medline were:
1. (ranibizumab or lucentis or bevacizumab or avastin).tw.
2. randomized controlled trial.pt.
3. controlled clinical trial.pt.
4. (randomly or randomised or randomized).tw.
5. 2 or 3 or 4
6. 1 and 5
7. (diabet* adj2 macular adj2 (edema or oedema)).tw.
8. diabetic maculopathy.tw.
9. 7 or 8
10. 6 and 9
These terms were adapted as appropriate for the other databases (for full details of the literature searches, see appendix 1 in the linked data supplement on bmj.com).
In addition, searches of clinicaltrials.gov and the European Union Clinical Trials Register were done for unpublished studies. One trial comparing ranibizumab and bevacizumab for diabetic macular oedema was identified.12 The authors were contacted, and we were informed that the study was currently recruiting participants. We also searched meeting abstracts—including those of the Association for Research in Vision and Ophthalmology, American Diabetes Association, and European Association for the Study of Diabetes—from 2002 to November 2011. No new studies were found.
To meet the inclusion criteria, studies were required to be randomised, include patients with diabetic macular oedema, evaluate ranibizumab or bevacizumab in one intervention arm, and measure best corrected visual acuity. We excluded trials in which one or more arms was undergoing surgical procedures such as cataract removal. Article titles were screened for eligibility by two reviewers, and abstracts or full texts were reviewed as necessary.
Studies were assessed for common comparators. A common comparator is a study arm that is similar in more than one trial and can therefore be used to connect trials. Using common comparators, we created several networks. Studies selected for potential inclusion in the network were assessed for similarity in four criteria: baseline patient population, administration and frequency of common comparator, outcomes assessment, and length of follow-up. Only one network was found to be methodologically suitable after assessing for similarity. The common comparator linking intervention arms was multiple laser photocoagulation. Bevacizumab 1.25 mg and ranibizumab 0.5 mg were included in the network, along with laser therapy. Other doses of bevacizumab and ranibizumab were considered, but the aforementioned doses were chosen as they are used in clinical practice.
Data from suitable trials were extracted by one author and checked by a second (JAF and DS). The two authors agreed, so it was not necessary to involve a third adjudicating author. Data extracted included study details, inclusion and exclusion criteria, baseline patient characteristics, dose, follow-up, change in best corrected visual acuity, and change in central macular thickness. In the event that salient data (such as standard deviations) were missing, study authors were contacted. Potential for bias was assessed using the Cochrane risk of bias tool.
The primary outcome measure was the proportion of patients with an improvement in best corrected visual acuity of more than two lines (or 10 letters) on the Early Treatment Diabetic Retinopathy Study (ETDRS) scale. Secondary outcomes were mean change from baseline in best corrected visual acuity (logarithm of the minimum angle of resolution (logMAR)), mean change from baseline in central macular thickness and adverse events. Best corrected visual acuity data was converted to logMAR units, a linear scale of visual acuity with positive values representing increasing visual loss. Reporting of results follows preferred reporting items for systematic reviews and meta-analyses guidelines.15
After data extraction and assessment of risk of bias, we carried out meta-analyses of the available direct evidence for each outcome, pooling pairwise comparisons between laser and either bevacizumab or ranibizumab using Review Manager software. The results were used for descriptive purposes and also for assessing heterogeneity and potential inconsistency with the indirect evidence.
An indirect comparison of bevacizumab 1.25 mg alone versus ranibizumab 0.5 mg alone was then performed for each outcome using WinBUGS Bayesian Markov chain Monte Carlo software.16 This involved taking direct evidence from published reports of trials that had compared either bevacizumab or ranibizumab with laser therapy (the common comparator) and entering the data into a simulation model to estimate the distribution of treatment effects that would be expected if a large number of head to head trials were to be conducted. Applying methods described by Lu and Ades17 and Dias,18 we estimated a treatment effect to be the median value from the posterior distribution of odds ratios generated from a random effects simulation model, assuming a level of between-study heterogeneity observable in the available data. In each analysis, two Markov chains were used with 20 000 iterations (following a “burn-in” of 10 000 iterations). We derived 95% credible intervals from the 2.5 and 97.5 centiles of the posterior distributions. If a 95% credible interval crossed the line of no effect (that is, it included a value of 1 for odds ratios or 0 for differences in means), then the result was interpreted as being non- significant.
Assumptions relating to heterogeneity and consistency were assessed using methods described by Song et al.19 Differences in follow-up periods between studies were addressed in the analysis of the proportion of patients with improved vision, by fitting a binomial likelihood model with a complementary log-log link function which treated study length as an additional rate parameter.18 Normal likelihood models with identity link functions were used in the comparisons of best corrected visual acuity and central macular thickness to calculate mean differences between interventions.
Secondary analyses compared bevacizumab alone with ranibizumab plus prompt laser or ranibizumab plus deferred laser. These analyses were conducted using the same methods as the primary analyses, with the exception that a logit link function was used in the proportion models because follow-up was similar.
Our initial aim was to include a comparison of each anti-vascular endothelial growth factor drug alone and in combination with laser. However, the included studies allowed comparison of only bevacizumab alone, ranibizumab alone, and ranibizumab plus laser. There was no direct evidence that compared bevacizumab plus laser with any comparator, so it could not be included in the analysis.
The results of the literature search are shown in fig 1⇑. Details of the included studies20 21 22 23 24 25 26 27 28 29 are shown in table 1⇓, and details of the excluded studies30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 are given in appendix 2 of the data supplement. Only five trials (reported in 10 published articles) were sufficiently homogenous with a common comparator to be included in the indirect comparison. Methodological heterogeneity was caused by either different sample populations (such as patients with previous failed laser therapy37) or different treatments (such as post-cataract surgery42 45 or quantity of laser treatments39 40). Nine studies could not be fitted into the network diagram because of a lack of common comparator.37 38 39 40 41 42 43 44 45
The final network diagram is shown in fig 2⇓. Two studies compared bevacizumab with laser therapy,20 23 two studies compared ranibizumab with laser therapy,25 26 and one study compared ranibizumab plus prompt or deferred laser therapy with laser therapy.29 Follow-up in three studies was 12 months, and in two studies was six months. Best corrected visual acuity was the primary outcome in all five studies. Tables 2⇓ and 3⇓ show the results from each included study for our primary and secondary outcomes. Table 4⇓ shows the results of separate meta-analyses before indirect comparison. The assumption of heterogeneity was shown to be appropriate in the posterior distribution, and no inconsistency was observed with the methods described by Song et al.47
Quality of studies
All studies included were of good quality (table 5⇓). Sequence generation was appropriate in all studies except in the READ-2 study,26 where the method was unclear. Allocation concealment was described in only one study,23 In the other four studies it was unclear. Three studies were appropriately masked,23 25 26 the remaining two20 28 were not because of the impracticality of masking patients to laser photocoagulation. Only one study failed to address incomplete data outcomes.25 Four studies used an intention to treat method. In one study a per protocol method was used.23 All studies were judged to be of low risk of bias from selective reporting because it was clear from the published articles that all main pre-specified outcomes were reported.
Best corrected visual acuity
As shown in table 6⇓, the proportion of patients with an improvement of more than two lines on the Early Treatment Diabetic Retinopathy Study (ETDRS) scale seemed comparable in both the bevacizumab and ranibizumab groups (odds ratio 0.95 (95% credible interval 0.23 to 4.32)). The differences between bevacizumab and ranibizumab with prompt or deferred laser seemed to favour the bevacizumab group but were also statistically non-significant (prompt laser addition odds ratio 0.80 (0.19 to 3.11), deferred laser addition odds ratio 0.61 (0.12 to 2.84)). However, wide credible intervals cannot exclude the possibility that one drug is superior.
As shown in table 7⇓, the mean change in best corrected visual acuity seemed to favour bevacizumab, but again this was not statistically significant (treatment effect −0.08 logMAR units (−0.19 to 0.04)), so superiority for either drug cannot be excluded. The addition of laser did not provide additional benefit to the ranibizumab group (prompt laser addition −0.10 units (−0.22 to 0.00), deferred laser addition treatment effect −0.10 (−0.23 to 0.03)).
Central macular thickness
Similarly, table 8⇓ shows that there was no statistically significant difference between ranibizumab and bevacizumab for mean change in central macular thickness. The difference in central macular thickness seemed greater in the bevacizumab group compared with ranibizumab alone (treatment effect −6.9 μm (−88.5 to 65.4)), but not with the addition of laser. However, none of these results was statistically significant, and the wide credible intervals cannot exclude a greater improvement, or worse outcome, for either drug.
Assessment of adverse events shows no consistent increase in adverse events in either group (table 9⇓). The DRCRN 2010 trial,28 the trial with the longest follow-up, reported more cardiovascular events in the sham injection plus laser therapy group than in the ranibizumab group (11.5% v 5.1%). In a two year retrospective study of bevacizumab, hypertension was slightly more prevalent in the ranibizumab and bevacizumab arms in the DRCRN 201028 and BOLT studies.21 However, hypertension was more common in the control arm in the DRCRN 2007 study.39 Endophthalmitis was a rare adverse event but was slightly more common in the intervention arms, apart from in the DRCRN 2010 trial,28 where endophthalmitis was more common in the control arm.
This is the first study to compare bevacizumab and ranibizumab for treatment of diabetic macular oedema in an indirect comparison. Indirect comparisons are subject to potential biases and should be interpreted with caution. There was no evidence of a difference in effectiveness between bevacizumab and ranibizumab when measured by proportion of patients who improved by more than two lines on the ETDRS scale, mean change in best corrected visual acuity measured in logMAR units, or mean change in central macular thickness. Included studies were of low statistical power because of small numbers of participants. This coupled with moderate heterogeneity between studies resulted in wide credible intervals around estimates of treatment effects.
Strength and limitations of study
There are strengths and weaknesses of indirect comparisons.63 In the absence of head to head trials, an indirect comparison is the best we can do to estimate the treatment effect between two interventions, albeit with greater uncertainty than in direct head to head randomised controlled trials.63
The need for a common comparator and similarity in design resulted in 12 trials being excluded from the indirect comparison. In any meta-analysis there is a trade-off between reducing heterogeneity between studies and including enough studies to be able to draw meaningful conclusions. We excluded all studies that were not similar. This resulted in a more robust network but fewer trials. This can result in reduced precision in indirect comparisons. Additional studies could have been added, but this would have increased heterogeneity, and we felt this would have compromised the validity of our results.
There were differences between study populations. Three ranibizumab trials included patients who had either been treated or not treated previously with laser therapy.25 26 28 One bevacizumab trial21 included patients who had been treated previously with laser therapy, and the other bevacizumab trial23 included patients who were laser naïve. By pooling the last two trials, a group of laser experienced and laser naïve patients was created. In addition, studies differed in the management of patients with two eligible eyes. The Soheilian study21 included both eyes: BOLT,20 RESTORE,25 and READ-226 studies included the worse eye; and DRCRN28 randomised both eyes, ensuring one eye was assigned to the control group. Further minor differences were the size of population, the length of follow-up in each trial, and the use of sham injections. Follow-up varied between six and 12 months. Three studies used sham injections alongside laser, whereas two studies used laser alone (see table 1⇑).
Assessment of the indirect comparison models showed that they were all suitable. Diagnostics, including inspection of Brooks-Gelman plots, indicated appropriate convergence and low levels of autocorrelation. Residual deviance statistics indicated that the models fitted well. Heterogeneity was observed in some of the pairwise meta-analyses, and so moderate heterogeneity was assumed in the indirect comparisons. The posterior distributions for between study variance indicated that this heterogeneity assumption was reasonable. The importance of checking for consistency between direct and indirect evidence in indirect comparisons has also been highlighted by a recent review, which indicated that inconsistency may be more common than previously observed.64 However, for all outcomes in the study, there was no statistically significant inconsistency between the indirect estimates and the available direct evidence.
It could be argued that the number of patients treated with bevacizumab was insufficient to provide safety data. However, a survey reported by Fung et al65 presented data on 7113 injections in 5228 patients from 70 centres in 12 different countries. The commonest adverse events were rare and included corneal abrasion (0.15%), mild ocular discomfort (0.14%), inflammation or uveitis (0.14%), and blood pressure increase (0.21%). The authors concluded that bevacizumab was not associated with an increase in adverse events.
Comparison with other studies
There are currently no randomised studies comparing bevacizumab and ranibizumab for the treatment of diabetic macular oedema. However, three randomised controlled trials comparing ranibizumab and bevacizumab in age related macular degeneration have recently been published.66 67 68 These found similar efficacy with ranibizumab and bevacizumab. The CATT trial67 reported more systemic adverse events in the bevacizumab group (risk ratio 1.29 (95% confidence interval 1.01 to 1.66)). The IVAN trial (n=610) found a lower incidence of heart failure or arteriothrombolic events in the bevacizumab group (odds ratio 0.23 (0.05 to 1.07), P=0.03) and no difference in serious adverse events (odd ratio 1.35 (0.80 to 2.27), P=0.25).68
Campbell et al conducted a population based nested case-control study of 91 378 older adults with a history of physician diagnosed retinal disease.69 The authors found that neither ranibizumab nor bevacizumab was associated with significant risks of ischaemic stroke, acute myocardial infarction, congestive heart failure, or venous thromboembolism. In an observational study, Ladas et al70 retrospectively compared 450 patients with diabetic macular oedema who received either bevacizumab or ranibizumab (1275 injections and 725 injections respectively) and found no difference in ocular or non-ocular events.
Meaning of the results
Caution is needed when interpreting the results since the small number of studies resulted in wide credible intervals and reduced precision. Our results indicate no difference in effectiveness between ranibizumab and bevacizumab. The number of patients found to have a gain of two or more lines on the ETDRS scale was similar with bevacizumab and with ranibizumab. Mean change in best corrected visual acuity non-significantly favoured bevacizumab.
Assessment of adverse events shows similar incidences between drugs. Bevacizumab has been shown to increase the risk of cardiovascular events when used systemically in colorectal cancer.71 72 However, far smaller doses are used intraocularly, and little reaches the systemic circulation. Cardiovascular disease is of particular importance in diabetes. However, it has been suggested that among patients treated with ranibizumab for age related macular degeneration, those with diabetes are at no higher risk than others.73 In a two year retrospective study of bevacizumab treatment for diabetic macular oedema, Arevalo et al30 found the rate of cardiovascular events to be only 1.7%. Although cardiovascular events do not appear to be increased with intravitreal anti-vascular endothelial growth factor drugs, large scale safety studies are still needed. It should also be noted that most trials exclude patients with recent cardiovascular events. Endophthalmitis secondary to anti- vascular endothelial growth factor treatment is a rare event, as shown in table 9⇑.
The anti-vascular endothelial growth factor drugs represent a significant advance in the treatment of diabetic macular oedema. The implication of this comparison for policy makers and clinicians is that there is no evidence from which to infer superiority of ranibizumab over bevacizumab in diabetic macular oedema, and therefore it is unlikely that ranibizumab would be cost effective compared with bevacizumab. The National Institute of Health and Clinical Excellence (NICE) concluded that ranibizumab was not cost effective compared with laser therapy.74 It is likely that bevacizumab would be cost effective compared with laser. However, since bevacizumab is not licensed for the treatment of diabetic macular oedema and there is a licensed alternative (ranibizumab), General Medical Council guidance would not recommend the use of bevacizumab.75 Clinicians and policy makers face a dilemma of using an unlicensed but clinically effective and probably cost effective treatment compared with an expensive alternative with similar outcomes. If the second option is chosen, the extra cost to the National Health Service in England alone may be in the order of £400m ($630m, €510m), which would be taken away from other groups of patients.76
We recommend that there should be an independent, head to head trial of ranibizumab and bevacizumab for treating diabetic macular oedema. This should be long enough to answer questions about the duration of treatment and should examine the place of laser therapy in the treatment pathway.
What is already known on this topic
The anti-vascular endothelial growth factor drugs ranibizumab and bevacizumab have been shown to be effective in the treatment of diabetic macular oedema
No head to head trial has been done, so their relative effectiveness is not known
What this study adds
This indirect comparison has found no evidence to suggest a difference in effectiveness between bevacizumab and ranibizumab
However, wide credible intervals cannot exclude a greater improvement, or worse outcome, for either drug
Cite this as: BMJ 2012;345:e5182
We thank the study authors who provided additional information: Wen Xing, Catey Bunce, and Michel Michaelides for the BOLT study; Peter Campochiaro and Afsheen Khwaja for READ-2; Mehdi Yaseri and Masoud Soheilian for Soheilian et al. We also thank Graeme MacLennan, Health Services Research Unit, University of Aberdeen, for his statistical advice.
Contributors: All authors contributed to the design, interpretation, and writing of the manuscript and approved the final manuscript. PR carried out the searches. JAF, DS, and PR screened titles and extracted data. AE carried out the indirect comparison. JAF is the guarantor. All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding: No external funding required.
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: the authors had support from University of Aberdeen and Warwick University for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; JAF, DS, PR and NW have undertaken an Evidence Review Group report for NICE on ranibizumab for diabetic macular oedema, which may be relevant to the submitted work.
Ethical approval: Not required
Protocol: No protocol exists for this study. The study arose out of a single technology appraisal for NICE.
Data sharing: Further details available in the ERG report on the NICE website (www.nice.org.uk/nicemedia/live/13125/53408/53408.pdf).
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.