- Dirk Eyding, project manager1,
- Monika Lelgemann, senior researcher2,
- Ulrich Grouven, statistician34,
- Martin Härter, head of department of medical psychology5,
- Mandy Kromp, statistician3,
- Thomas Kaiser, head of department of drug assessment3,
- Michaela F Kerekes, data manager3,
- Martin Gerken, researcher6,
- Beate Wieseler, deputy head of department of drug assessment3
- 1German Cancer Society, Berlin, Germany (former affiliation: Institute for Quality and Efficiency in Health Care)
- 2Medical Advisory Service of Social Health Insurance, Essen, Germany
- 3Institute for Quality and Efficiency in Health Care, Cologne, Germany
- 4Hanover Medical School, Hanover, Germany
- 5University Clinic Eppendorf, Hamburg, Germany
- 6Health Technology Assessment Centre, University of Bremen, Bremen, Germany
- Correspondence to: B Wieseler, Institute for Quality and Efficiency in Health Care, Dillenburger Strasse 27, 51105 Cologne, Germany
- Accepted 16 August 2010
Objectives To assess the benefits and harms of reboxetine versus placebo or selective serotonin reuptake inhibitors (SSRIs) in the acute treatment of depression, and to measure the impact of potential publication bias in trials of reboxetine.
Design Systematic review and meta-analysis including unpublished data.
Data sources Bibliographic databases (Medline, Embase, PsycINFO, BIOSIS, and Cochrane Library), clinical trial registries, trial results databases, and regulatory authority websites up until February 2009, as well as unpublished data from the manufacturer of reboxetine (Pfizer, Berlin).
Eligibility criteria Double blind, randomised, controlled trials of acute treatment (six weeks or more) with reboxetine versus placebo or SSRIs in adults with major depression.
Outcome measures Remission and response rates (benefit outcomes), as well as rates of patients with at least one adverse event and withdrawals owing to adverse events (harm outcomes).
Data extraction and data synthesis The procedures for data extraction and assessment of risk of bias were always conducted by one person and checked by another. If feasible, data were pooled by meta-analyses (random effects model). Publication bias was measured by comparing results of published and unpublished trials.
Results We analysed 13 acute treatment trials that were placebo controlled, SSRI controlled, or both, which included 4098 patients. Data on 74% (3033/4098) of these patients were unpublished. In the reboxetine versus placebo comparison, no significant differences in remission rates were shown (odds ratio 1.17, 95% confidence interval 0.91 to 1.51; P=0.216). Substantial heterogeneity (I2=67.3%) was shown in the meta-analysis of the eight trials that investigated response rates for reboxetine versus placebo. A sensitivity analysis that excluded a small inpatient trial showed no significant difference in response rates between patients receiving reboxetine and those receiving placebo (OR 1.24, 95% CI 0.98 to 1.56; P=0.071; I2=42.1%). Reboxetine was inferior to SSRIs (fluoxetine, paroxetine, and citalopram) for remission rates (OR 0.80, 95% CI 0.67 to 0.96; P=0.015) and response rates (OR 0.80, 95% CI 0.67 to 0.95; P=0.01). Reboxetine was inferior to placebo for both harm outcomes (P<0.001 for both), and to fluoxetine for withdrawals owing to adverse events (OR 1.79, 95% CI 1.06 to 3.05; P=0.031). Published data overestimated the benefit of reboxetine versus placebo by up to 115% and reboxetine versus SSRIs by up to 23%, and also underestimated harm.
Conclusions Reboxetine is, overall, an ineffective and potentially harmful antidepressant. Published evidence is affected by publication bias, underlining the urgent need for mandatory publication of trial data.
Reboxetine, the first selective norepinephrine (noradrenaline) reuptake inhibitor used in the treatment of depression,1 mainly acts by binding to the norepinephrine transporter and blocking reuptake of extracellular norepinephrine.2 The drug is “indicated for the acute treatment of depressive illness or major depression and for maintaining the clinical improvement in patients initially responding to treatment.”3 Reboxetine has been approved for marketing in many European countries (for example, the United Kingdom and Germany) since 1997. In the United States, however, the application for approval was ultimately rejected after preliminary acceptance.2 4
Compared with the overall amount of antidepressants prescribed, reboxetine’s share is relatively small. For example, of 974 million defined daily doses of antidepressants prescribed in Germany in 2008, reboxetine accounted for 6.7 million defined daily doses.5 The average cost of reboxetine per defined daily dose was €1.87 (£1.54; $2.39) for Edronax (Pfizer, Berlin) to €2.09 for Solvex (Merz, Frankfurt), compared with €0.52 for selective serotonin reuptake inhibitors (SSRIs), the most commonly prescribed antidepressants.5
Although reboxetine has been claimed to show superior efficacy to placebo and similar efficacy to other antidepressants,1 6 7 8 9 10 the clinical relevance of the drug has been queried. A recent systematic review by Cipriani et al11 included a network meta-analysis of active controlled trials and found that reboxetine was not only significantly less effective than the other newer antidepressants investigated, but was also the drug with the highest dropout rates.
The German Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG)) conducted a health technology assessment of the short term and long term benefits and harms of reboxetine, bupropion, and mirtazapine in placebo controlled and active controlled trials of adult patients with major depressive disorder. Both published and previously unpublished data were considered. The full German language report and an English summary are available on the institute’s website.12 13 The responsibilities and methodological approach of IQWiG are described in its methods paper online.14
This publication presents the main findings of the reboxetine trials with the aim of assessing the benefits (remission and response rates) and harms (rates of patients with at least one adverse event and rates of withdrawals owing to adverse events) of reboxetine versus placebo or SSRIs in the acute treatment of major depressive disorder. In addition, for the present paper we assessed potential publication bias by comparing results from published and unpublished trials of reboxetine.
We developed and followed a standardised protocol for all steps of the review.15
The health technology assessment report that formed the basis of this publication included both published and unpublished trials of reboxetine that had the following characteristics:
Double blind, randomised controlled design
Investigation of adult patients with major depressive disorder as their primary diagnosis according to the International Classification of Diseases, the Diagnostic and Statistical Manual of Mental Disorders, or the Research Diagnostic Criteria
Acute treatment (at least six weeks duration) or long term treatment (at least six months (relapse) or 12 months (recurrence)) for prevention of relapse or recurrence
Comparison of reboxetine with placebo or any antidepressant (including St John’s wort); treatment according to approval status in Germany
Evaluation of at least one prespecified patient relevant outcome (in this context, the term “patient relevant” refers to “how a patient feels, functions, or survives”16)
Publication in English, German, or French (or any other language if the trial was classified as potentially relevant according to the English title or abstract)
Availability of a full text document (for example, journal article or clinical study report).
This publication is limited to acute treatment trials of reboxetine versus placebo or SSRIs. The outcomes presented are restricted to the most commonly reported outcomes in depression trials. Benefit outcomes were remission and response rates. Harm outcomes were rates of patients with at least one adverse event (any adverse event according to the definitions used in the primary trials) and rates of withdrawals owing to adverse events (any adverse event according to the definitions used in the primary trials). Harms were further described by the overall rates of patients with serious adverse events (any serious adverse event according to the definitions used in the primary trials).
According to the review protocol, response and remission data were analysed on the basis of the definitions and instruments used in the primary trials. All trials applied the Hamilton depression rating scale and 10 trials additionally applied the Montgomery-Åsberg depression rating scale. We primarily considered the response and remission outcomes on the Hamilton depression rating scale. In all trials, response was defined as a reduction in the score on the Hamilton depression rating scale of 50% or more from baseline to end of study, and remission was defined as a reduction in the score on the Hamilton depression rating scale to below an absolute threshold at end of study (score ≤10 in all trials except in one trial where the score threshold was ≤8).
Search strategy and study selection
We searched for relevant primary and secondary publications (systematic reviews and health technology assessment reports) in Medline, Embase, PsycINFO, BIOSIS, and the Cochrane Library published up until February 2009. The full search strategy, including the search terms used for the various databases, has been described elsewhere.12
We scrutinised the reference lists of the primary and secondary publications retrieved to identify further trials. In addition, clinical trial registries and trial results databases available on the internet were screened, as were the websites of the European Medicines Agency and the US Food and Drug Administration.
In order to obtain the most complete dataset possible, we asked the manufacturer of reboxetine (Pfizer) to supply unpublished trials and additional unpublished data from published trials. As a prerequisite for the use of unpublished data, IQWiG asked the manufacturer to sign an agreement requiring: (1) submission of a list of all sponsored published and unpublished trials investigating reboxetine; (2) submission of documents (generally the clinical study reports) compliant with the CONSORT criteria for all relevant trials selected by IQWiG; and (3) permission for publication of all previously unpublished relevant data. This procedure was required to avoid bias through selective provision of data. Finally, people and parties who had submitted comments on the preliminary version of the health technology assessment report at the public hearing in July 2009 were asked to provide any additional relevant trials.
Two reviewers independently screened titles and abstracts of the retrieved citations to identify potentially eligible primary and secondary publications. In a first broad screening step, citations were excluded if clearly irrelevant; that is, if a primary publication was not a clinical trial in humans with depression, or if a secondary publication of eligible trials was not a systematic review. In a second screening step, the full set of eligibility criteria was applied. Potentially relevant articles were then screened as full texts. Disagreement was resolved by consensus.
Data extraction and assessment of risk of bias
The individual steps of the data extraction and assessment of risk of bias were always conducted by one person and checked by another. Details of the trials were extracted using standardised tables. Information and data from publications were supplemented by clinical study reports provided by the manufacturer. We always extracted data from the intention to treat populations. Clinical study reports were always considered the primary source in instances of conflict with the publication. Disagreement was resolved by consensus.
Information was extracted from each included trial on:
Study characteristics, including citation, study design, setting (inpatient or outpatient), inclusion and exclusion criteria, length of follow-up, sample size, location, number of centres, and year of completion
Characteristics of the study participants, including age, gender, and disease severity at baseline
Characteristics of the test and control interventions, including dose
Outcomes and type of outcome measures (outcomes as presented above; measurement tools as used in the individual trials)
Risk of bias items.
The risk of bias at the study level was assessed on the basis of the adequacy of the following criteria: randomisation; allocation concealment; blinding of patients and investigators; and complete and non-selective results reporting. The risk of bias at the outcome level was assessed on the basis of the adequacy of: application of the intention to treat principle; blinding of the outcome assessor; statistical evaluation; and complete and non-selective results reporting. Trials and outcomes were categorised into those with a low risk of bias and those with a high risk.
If feasible and meaningful, data were pooled by means of meta-analyses. Effect measures were reported as odds ratios (ORs) and 95% confidence intervals (CIs) for binary data. A random effects model was used to calculate a pooled effect estimate. Statistical significance was assumed for P<0.05. Heterogeneity of effect sizes was assessed by using the I2 statistic; pooled estimates were not calculated if substantial heterogeneity was observed (I2>50%). If heterogeneity with I2>50% was shown, sensitivity analyses were conducted, when appropriate, to assess possible sources of heterogeneity across the trials included. The review protocol prespecified potential effect modifiers, including gender and trial setting (inpatient or outpatient). These factors were investigated by means of random effects meta-regression analyses based on aggregate study data.17
To assess publication bias, effect sizes in the published, unpublished, and overall dataset were compared. In addition, the differences in effect sizes between published and unpublished data, and between published and overall data, were expressed as the ratio of odds ratios (ROR). The magnitude of the overestimation or underestimation of effect sizes in published versus overall data (publication bias) was expressed as percentage changes.
Meta-analyses were performed using SAS version 9.1.3. If meta-analyses were not possible, the results of the individual trials were assessed.
The process of study selection is presented in figure 1⇓. The search in bibliographic databases yielded 2596 citations, of which 713 were classified as potentially relevant and subjected to strict eligibility assessment. A total of 13 citations (10 trials) met the inclusion criteria; however, two of these 13 citations were publications on subgroups of otherwise unpublished trials,18 19 and one was the only available publication on the total population being studied but did not report the main outcomes.20 In the assessment of publication bias, we considered these three trials to be “unpublished.” No trials were identified in clinical trial or trial results registries or in the European Medicines Agency or FDA websites.
The retrieval of previously unpublished trials was hampered by the fact that during preparation of the preliminary health technology assessment report, the manufacturer of reboxetine did not provide a complete list of unpublished trials as requested by IQWiG.21 22 Secondary publications clearly indicated that further potentially relevant unpublished trials existed.6 8 As the preliminary report showed that reboxetine had been tested in at least 16 trials including about 4600 patients, but data on almost two thirds of these patients were not accessible, the institute initially concluded that no meaningful assessment of reboxetine was possible.21 22
After the publication of the preliminary report, the manufacturer decided to cooperate and provided most of the missing data (one venlafaxine controlled trial23 was not available as a full publication). Thus, an additional 10 previously unpublished or incompletely published reboxetine trials were considered in the final health technology assessment report.12 Two trials with tricyclic antidepressants as active controls and two relapse prevention trials were excluded from the present analysis.
Of the remaining 13 eligible acute treatment trials, three were placebo controlled, five were active controlled, and five had both placebo and active controlled arms (one of which had a tricyclic antidepressant arm that was not considered). A total of 4098 patients were analysed: 2256 in the reboxetine versus placebo comparisons and 2641 in the reboxetine versus SSRI comparisons.
The characteristics of the pool of 13 acute treatment trials that were placebo controlled, SSRI controlled, or both are presented in tables 1 and 2⇓ ⇓. All trials were sponsored by predecessors of Pfizer (Pharmacia, and Pharmacia & Upjohn), except for Berlanga and Flores-Ramos 2006 (sponsored by Lundbeck), and included adult patients with major depressive disorder according to the third edition, revised or the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders. In the four fluoxetine controlled trials and in one citalopram controlled trial, the SSRIs were potentially underdosed compared with reboxetine (according to doses standardised on the basis of the maximum approved dose; see table 2). The trials were well balanced between treatment arms with respect to patient baseline characteristics.
There were no major differences between trials in terms of dosage and mean patient age. However, there were differences in setting (inpatient, outpatient, or both) and baseline severity of depression as measured by the Hamilton depression rating scale. For more details on trial characteristics see web table A.
Risk of bias
The overall methodological quality of the trials was good (table 3⇓). At the trial level, the risk of bias was low in all but one study, which had a high risk of bias at the trial level owing to unclear allocation concealment and blinding. At the outcome level, the risk of bias was low for all four benefit and harm outcomes in nine out of the 13 trials. Three trials had a high risk of bias at the outcome level owing to an inadequate intention to treat analysis. Analyses excluding the outcomes at high risk of bias did not alter the conclusions (data not shown). As no clear dose-response relationship has been shown for fluoxetine and citalopram,24 25 the potential underdosing of these agents in five trials did not affect the risk of bias.
Owing to the availability of a comprehensive set of the relevant data on reboxetine versus placebo and SSRIs, the risk of publication bias on the results of the final analysis was minor.
Effects of interventions
In this text, the terms “superior” and “inferior” refer to statistically significant differences between treatment groups (P<0.05).
Meta-analyses of remission and response rates
The Hamilton depression rating scale was used in the meta-analyses of remission and response rates. No statistically significant difference between reboxetine and placebo was noted in the meta-analysis of remission rates (OR 1.17, 95% CI 0.91 to 1.51; P=0.216; fig 2⇓).
Substantial heterogeneity (I2=67.3%; p=0.003) was shown in the meta-analysis of response rates including all eight trials that compared reboxetine with placebo, and consequently no point estimate was calculated. The only known inpatient trial—trial 091 (n=52), which had an OR of 11.43 (95% CI 3.10 to 42.12)—was obviously a statistical outlier (figure 2).
In the sensitivity analysis using meta-regression analysis, setting had an effect on the outcome response. Patients who received reboxetine in an inpatient setting were more likely to show a good response compared with placebo than were patients who received reboxetine in an outpatient setting (P=0.001 inpatients v outpatients; trials 091 v 049 and 050). In a second scenario, the proportion of inpatients was used as the independent variable. This analysis also included trials 014 and 015, for which the proportion of inpatients was available from Montgomery et al 2003.7 This scenario confirmed the influence of setting (P<0.001). The meta-analysis of response rates in the outpatient only trials (049 and 050) showed no statistically significant difference between reboxetine and placebo (OR 1.05, 95% CI 0.73 to 1.50; P=0.796 I2=0%). These findings indicate that patient setting was the most probable effect modifier. After exclusion of trial 091, the meta-analysis of response rates in the seven remaining trials showed no statistically significant difference between reboxetine and placebo (OR 1.24, 95% CI 0.98 to 1.56, P=0.071, I2=42.1%; figure 2).
Reboxetine was inferior to SSRIs in the meta-analysis of remission rates (OR 0.80, 95% CI 0.67 to 0.96; P=0.015; fig 3⇓). A similar, although non-significant, trend in remission rates was shown when reboxetine was compared with the individual SSRIs (fluoxetine, paroxetine, and citalopram). However, if remission rates according to the Montgomery-Åsberg depression rating scale rather than the Hamilton depression rating scale were analysed from trials using this instrument as the primary scale (trials 046 and 047), reboxetine was inferior to paroxetine (OR 0.72, 95% CI 0.56 to 0.93). In the long term acute treatment trial (trial 043), reboxetine was inferior to citalopram (OR 0.51, 95% CI 0.32 to 0.83). However, the intention to treat principle was violated in this trial, so a worst case analysis was conducted in which the difference in remission rate compared with citalopram was no longer statistically significant.
Reboxetine was also inferior to SSRIs in the meta-analysis of response rates (OR 0.80, 95% CI 0.67 to 0.95; P=0.010). A similar trend was shown when reboxetine was compared with the individual SSRIs, where the trend reached statistical significance in the comparison of reboxetine and paroxetine (OR 0.79, 95% CI 0.64 to 0.99; P=0.04). In the long term acute treatment trial, no statistically significant difference was shown between reboxetine and citalopram (OR 0.60, 95% CI 0.35 to 1.03).
The overall findings were also reflected in the subset of trials that were both placebo controlled and SSRI controlled (n=4), which are suited to demonstrating assay sensitivity. The SSRIs were superior to placebo and reboxetine in this analysis, but no statistically significant difference was shown between reboxetine and placebo (see web figure A).
Meta-analyses of adverse events and withdrawals owing to adverse events
Reboxetine was inferior to placebo (that is, it was associated with higher event rates) in the meta-analyses of the rates of patients with at least one adverse event and in the meta-analysis of the rates of withdrawals owing to adverse events (OR 2.14, 95% CI 1.59 to 2.88; P<0.001 and OR 2.21, 95% CI 1.45 to 3.37; P<0.001, respectively; fig 4⇓).
The rates of patients with at least one adverse event did not differ significantly between patients treated with reboxetine and those who received an SSRI (OR 1.06, 95% CI 0.82 to 1.36; P=0.667; fig 5⇓). The same finding was seen for patients on reboxetine versus those treated with individual SSRIs. A meta-regression analysis showed a gender effect in the comparison of reboxetine with fluoxetine (P=0.022 for the interaction test): in men reboxetine was inferior to fluoxetine in the meta-analysis of patients with at least one adverse event (OR 2.76, 95% CI 1.28 to 5.93), whereas no significant difference was shown in women (OR 0.90, 95% CI 0.51 to 1.59). In the long term acute treatment trial, reboxetine was inferior to citalopram (OR 1.57, 95% CI 1.03 to 2.38).
Substantial heterogeneity (I2=67.4%) was shown in the meta-analysis of the rates of withdrawals owing to adverse events in the comparison between reboxetine and SSRIs, which was in part owing to variations in the results of the individual SSRIs. The comparison between reboxetine and fluoxetine showed low heterogeneity (I2=19.3%) and statistically significantly more withdrawals owing to adverse events for reboxetine (OR 1.79, 95% CI 1.06 to 3.05; P=0.031). On the other hand, the comparison between reboxetine and paroxetine showed substantial heterogeneity (I2=84.2%), but the sensitivity analysis did not identify a potential effect modifier. We therefore concluded that there was no proof of a difference between reboxetine and paroxetine concerning rates of withdrawals owing to adverse events. In the long term acute treatment trial, reboxetine was inferior to citalopram (OR 4.61, 95% CI 2.15 to 9.89).
Further information on adverse events
The rates of serious adverse events (including events related to suicide) were low and did not differ significantly between reboxetine and placebo or reboxetine and SSRIs (data on overall serious adverse events not shown). A total of 18 serious adverse events related to suicide (suicidal tendencies, suicide attempts, or completed suicides) were noted (six for reboxetine; four for placebo; eight for SSRIs). One death (a completed suicide under placebo) was reported, which was the only mortality in the study arms investigated. However, with respect to study design and duration, none of the trials were aimed at investigating suicide related events or overall mortality. The validity of the results of these outcomes is therefore limited and the data do not provide clarification.
A substantial proportion of patient data (74%) had not been previously published: 86% (1946/2256 patients) in the comparisons of reboxetine and placebo and 67% (1760/2641 patients) in the comparisons of reboxetine and SSRIs (table 1⇑).
For both benefit outcomes, the addition of unpublished data changed the superiority of reboxetine versus placebo shown in published data to a non-significant difference and also changed the non-significant difference between reboxetine and SSRIs to an inferiority of reboxetine (fig 6⇓). Comparison of the published data with the full dataset (published and unpublished) showed that the published data overestimated the beneficial effect of reboxetine compared with placebo by 99-115% and of reboxetine compared with SSRIs by 19-23%.
For both harm outcomes, the addition of unpublished data changed the non-significant difference between reboxetine and placebo shown in published data to an inferiority of reboxetine. For rates of withdrawals owing to adverse events, the addition of unpublished data changed the non-significant difference between reboxetine and fluoxetine to an inferiority of reboxetine; this was primarily owing to the increased power of the analysis rather than to major differences in withdrawal rates between published and unpublished data. For patients with at least one adverse event, no significant impact of unpublished data was shown in the comparison between reboxetine and SSRIs.
To our knowledge, this is the first systematic review of a comprehensive evidence base of published and unpublished acute treatment trials of reboxetine versus placebo or SSRIs in adults with major depressive disorder. We found that, overall, reboxetine was ineffective as an antidepressant because it showed no benefit over placebo and was inferior to SSRIs for remission and response rates. A benefit of reboxetine (higher response rates) was shown in a placebo controlled trial in inpatients; however, this trial was too small to draw general conclusions on the effect of reboxetine in this patient population. Reboxetine was inferior to placebo for both harm outcomes and to fluoxetine for rates of withdrawals owing to adverse events.
Given the potential underdosing of fluoxetine and citalopram in five trials, our findings on reboxetine might be considered conservative. At the same time, the advantages of SSRIs concerning harm might be overestimated. However, as stated, no clear dose-response relationship has been shown for fluoxetine and citalopram.24 25 Furthermore, in our test of assay sensitivity that included two of the four potentially underdosed fluoxetine arms, even the lower fluoxetine dose showed a clear benefit compared with placebo (OR 1.98, 95% CI 1.19 to 3.28, I2=53.8%), thus qualifying the effect of dosing in treatment for depression.
Data on 74% of the patients included in our analysis was unpublished, indicating that the published evidence on reboxetine so far has been severely affected by publication bias. Our comparison of published and unpublished trials confirmed this assumption: the positive benefit-risk ratio of reboxetine in the published literature was changed to a negative ratio if unpublished trials were added to the analysis.
Comparison with other reviews
The solely placebo controlled analyses by Ferguson et al6 and Montgomery et al7 both found greater efficacy (including higher response rates) for reboxetine compared with placebo, and Ferguson et al also found comparable harms. However, both reviews included only three of the eight studies considered in our review (plus the inpatient trial by Ban et al 199826). These two reviews also included only one unpublished trial (015), even though the relevant unpublished trials had been completed before publication of these analyses and both reviews were cowritten by a sponsor employee. The meta-analysis by Chuluunkhuu et al9 concluded that reboxetine showed superior efficacy to placebo and found no difference in efficacy of reboxetine compared with SSRIs and other antidepressants. However, this analysis considered only published data.
Although the meta-analysis by Papakostas et al8 identified and included a large body of unpublished studies that used SSRIs as the control (the same set as we used), they found no significant difference in response rates between SSRIs and reboxetine (risk ratio 1.08, 95% CI 0.98 to 1.19). Their analysis included the long term acute treatment trial 043, which we analysed separately. In contrast, our analysis showed that reboxetine was inferior to SSRIs, even if trial 043 was included (recalculated according to Papakostas: risk ratio (SSRI v reboxetine) 1.10, 95% CI 1.03 to 1.17; P=0.003). The reason for this discrepancy is unclear, because Papakostas et al reported only point estimates and CIs and did not report the number of actual events or the corresponding populations.
The widely discussed systematic review by Cipriani et al,11 which assessed 12 new generation antidepressants in a network meta-analysis and ranked reboxetine last, had similar findings to those of our review. These authors found significantly lower response rates for reboxetine than for all SSRIs investigated, as well as significantly higher dropout rates versus fluoxetine, citalopram, escitalopram, and sertraline. However, despite the similarity in findings, the evidence base of the Cipriani review differed markedly from that in our review because placebo controlled trials were omitted and trials that were not double blind, which carry a higher risk of bias, were considered. In addition, unpublished trials of reboxetine on file at the manufacturers were not considered, even though significant publication bias has been shown in antidepressant research. Given the sources of bias noted, the results of the Cipriani review should be interpreted with caution.
Our findings that reboxetine was superior (higher response rates) to placebo in a small trial in inpatients and that patient setting was a probable effect modifier are supported by the four week active controlled and placebo controlled inpatient trial by Ban et al26 (n=169 in the reboxetine and placebo arms), which we excluded owing to its short duration. Ban et al also found a statistically significant higher response rate in inpatients who received reboxetine compared with those who received placebo (60% v 35%; OR 2.70, 95% CI 1.45 to 5.03 (own calculation)).
Strengths and limitations of the review
The main strength of our review is the inclusion of a large amount of previously unpublished data. As we made extensive efforts to identify unpublished trials, we are optimistic that we analysed the vast majority or even all of the placebo controlled and SSRI controlled double blind randomised trials of reboxetine in adults with major depression.
Our review also has a number of limitations. We only had access to aggregated data. To assess the impact of effect modifiers, meta-analysis of individual patient data would be needed to determine the setting in studies with mixed settings and to test our hypothesis that the setting was the effect modifier explaining the substantial heterogeneity in the meta-analysis of response rates in placebo controlled trials.
Our results are further limited by the fact that they only refer to acute treatment trials, only one of which lasted more than eight weeks. However, six to eight weeks is the standard study duration in trials investigating the acute treatment of depression. The long term acute treatment trial showed similar, though not always statistically significant, trends to the short term trials. Other long term outcomes in depression, such as prevention of relapse or recurrence, were not the focus of this paper.
Finally, except for a subgroup analysis for gender and setting, we assessed total populations of patients with major depressive disorder. No analyses were performed in other subgroups of patients (for example, patients with severe disease or specific major depressive disorder symptoms such as anxiety or cognitive impairment), in which treatment effects may differ.
Our difficulties in retrieving unpublished trial data and our results of the comparison between published and previously unpublished trials are a further example of publication bias, a problem that has been known in clinical research for decades.27 28 29 30 31 A recent narrative review has shown that publication bias affects a wide range of medical indications and interventions.32 Such bias, including industry sponsorship bias, has frequently been identified in research on antidepressants (table 4⇓). For example, Turner et al33 published a comparison of FDA reviews of placebo controlled antidepressant trials and matching publications, which showed that, overall, published trials overestimated effect sizes by 32% (11 to 69% for individual agents); the estimates in our review were even higher. Whittington et al34 investigated SSRIs in the treatment of childhood depression and found that the addition of unpublished data reversed the benefit-risk profile for all but one SSRI.
In addition to publication bias, outcome reporting bias has been identified as a major problem in the reporting of clinical trials, resulting in a distorted public record of an intervention.35 36 37 38 Our review also identified this type of bias—for three reboxetine trials, only results on subpopulations or selected outcomes were available in the published literature (trials 047, 050, 052; table 1⇑).
The more positive benefit-risk ratio in published data compared with unpublished data also affects the content of clinical guidelines. For example, the National Institute for Health and Clinical Excellence (NICE) guideline on the treatment and management of depression in adults is based on published studies of reboxetine, and concludes that “Reboxetine is superior to placebo and as effective as other antidepressants in the treatment of depression.”10 In our opinion, this conclusion can no longer be upheld.
The ongoing problem of publication bias shows that unbiased decision making in health care requires mandatory public disclosure of all clinical trial data. The US FDA Amendments Act of 200739 solves the problem in part by requiring protocol information and study results for clinical trials to be made public on the clinicaltrials.gov website (www.clinicaltrials.gov; please see accompanying comment (doi:10.1136/bmj.c4942) for further details). Similar legislation is also being introduced in Europe, with the mandatory public disclosure of data from the clinical trials database EudraCT (eudract.ema.europa.eu),40 41 but the date of implementation is not yet clear.
As the full assessment reports on reboxetine prepared by regulatory authorities are not publicly available, it is not clear as to how the comprehensive body of evidence (including that on efficacy outcomes) generated after reboxetine was approved in Europe in the late 1990s has been analysed by these authorities. The reason for the difference in approval status of reboxetine between Europe and the US thus remains unclear.
Conclusions and policy implications
Our analysis of a comprehensive evidence base of published and unpublished trials of reboxetine compared with placebo or SSRIs in adults with major depressive disorder indicates that reboxetine is, overall, an ineffective and potentially harmful antidepressant. Published evidence on reboxetine has been substantially affected by publication bias, underlining the urgent need for mandatory publication of clinical trial data, including data on older agents.
What is already known on this topic
Reboxetine has been approved for the treatment of major depression in many European countries, but the application for approval was rejected in the United States
Doubts have been raised about the efficacy of reboxetine
Research into antidepressants is particularly affected by publication bias
What this study adds
Overall, reboxetine is not effective for the treatment of major depressive disorder
We found a higher rate of patients affected by adverse events than with placebo and higher withdrawal rates owing to adverse events than with placebo and fluoxetine
This meta-analysis provides a striking example of publication bias in which the previously favourable risk-benefit profile of reboxetine shown in published trials is reversed by the addition of unpublished data
Post-approval regulatory decisions (for example, reimbursement decisions based on the findings of health technology assessment reports) might be affected by publication bias
Our findings underline the need for mandatory publication of clinical trial results
Cite this as: BMJ 2010;341:c4737
We thank Katharina Quitmann, Yvonne-Beatrice Schüler, and Volker Vervölgyi for general support, and Natalie McGauran for editorial support. We also thank Elke Hausner for developing the search strategy and performing the literature search. All persons listed above are employed by the Institute for Quality and Efficiency in Health Care (IQWiG), Cologne, Germany.
Contributors: DE coordinated and BW supervised the review. DE, ML, UG, MH, TK, MG, and BW designed the protocol. UG and MK planned and did the statistical analyses. DE, ML, and MG selected studies. MFK did the data extraction. All authors (except MFK) assessed studies. DE wrote the first draft of the paper. All authors contributed to data interpretation and to critical revision of the paper, and have seen and approved the final version. BW is the guarantor.
Funding: This work was supported by IQWiG. The original health technology assessment report was commissioned by the German Federal Joint Committee. MG, MH, and ML acted as consultants for the preparation of the original health technology assessment report. For this they were reimbursed by IQWiG.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any company for the submitted work; DE was employed by H Lundbeck A/S, Copenhagen, between January 2006 and April 2007; MH received remuneration from Boehringer Ingelheim and Lilly Pharma for three talks on depression guidelines in 2008; and UG, MK, TK, MFK, and BW are employees of IQWiG. DE is a former employee of IQWiG. In order to produce unbiased health technology assessment reports, the institute depends on access to all of the relevant data on the topic under investigation. The authors therefore support the mandatory worldwide establishment of trial registries and study results databases. ML and MH were involved in the development of the German Disease Management Guideline on Depression.
Ethical approval: Not required.
Data sharing: The full (German) version of the health technology assessment report (including the search strategy) and the clinical study reports on reboxetine are available on the IQWiG website (www.iqwig.de).
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.