Antidepressant efficacy of agomelatine: meta-analysis of published and unpublished studiesBMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g1888 (Published 19 March 2014) Cite this as: BMJ 2014;348:g1888
All rapid responses
1. I endorse the principles of Retraction Watch
"retractions as a window into the scientific process"
2. I'll quote from a previous response posted today on the BMJ:
"I also take it very seriously when evidence does not support statements about an active pharmacological principle, especially when there are serious concerns about the safety and effectiveness of a molecule. The source of the misleading information (e.g., pharmaceutical company, peer-review medical journal, governamental, academic institution, etc) doesn't matter."
3. I'll quote from the previous response to this paper "In this first letter, I prefer to put emphasis on real data of published and unpublished agomelatine studies, allowing numbers to speak for themselves."
4. This paper doesn't meet the quality standards of the BMJ.
Question: any reason not to retract?
Competing interests: I support open data and transparency in research.
I read with great interest the article entitled: “Antidepressant efficacy of agomelatine: meta-analysis of published and unpublished studies” soon after this paper was published by the BMJ. Because I’m aware of the phenomenon of unpublished and misreported human research involving active pharmacological principles, I also downloaded data (March 28, 2014) of agomelatine study registries from ClinicalTrials.gov and WHO ICTRP. In this first letter, I prefer to put emphasis on real data of published and unpublished agomelatine studies, allowing numbers to speak for themselves.
1. 12 out of 20 studies included in the study by Taylor and colleagues are unregistered. This data represents evidence that calculations of unpublished and published studies in clinical trial registries could be an understimate because an unknown number of unpublished and unregistered studies.
2. There is also evidence of other agomelatine studies, not included in this meta-analysis, and absent from clinical trial registries.
3. Over 80% of agomelatine studies are unpublished.
4. Quote from the article published by Taylor and colleagues follows.
"A meta-analysis of published trials suggested robust efficacy in major depression, with an estimated effect size of 0.26 compared with placebo.3”
- Quote from the abstract in the cited reference (3) is shown below this line.
"Agomelatine (n=1274) stood superior to placebo (n=689) by a small margin (SMD -0.26, p=3.48×10-11) and the superiority of agomelatine (n=834, dose ≥ 25 mg/d) over antidepressants (paroxetine, fluoxetine, sertraline, venlafaxine; n=864) was even smaller (SMD -0.11, p=0.02). Although there is evidence of the superiority of agomelatine over placebo and selected antidepressants, it is questionable whether the magnitude of effect size is clinically significant and sample characteristics are relevant to the general patient population with major depressive disorder."
5. Recent marketing campaigns of Valdoxan (agomelatine) in Colombia involve giving free samples of agomelatine. It’s concerning the misleading advertising of Valdoxan as it can be seen in the following video presented during the launch to the market of Valdoxan in Central America.
6. Raw data is available
7. List of published and unpublished agomelatine study records
8. Attrition in agomelatine clinical trials is considerably high. Patient flow is available in some agomelatine studies evaluated by the EMA.
Competing interests: - I endorse the principles of open data in human biomedical research
We agree with very little of what Dr Singh puts forward
The interpretation of statistical and clinical significance is covered in the paper's Discussion section. Briefly: placebo is a powerful treatment in depression, more so than many 'active' medical treatments; agomelatine's superiority to placebo is clear, albeit modest, but its effect size is not notably different from other antidepressants or indeed widely used treatments in general medicine.
An allusion is made by Dr Singh to NICE's definition of clinical significance (three points on the Hamilton Depression Rating Scale) but this has no foundation other than its recommendation by NICE (who do not say how they derived this figure). We do not believe it is relevant to our discussion.
Direct comparison with other antidepressants revealed no suggestion of an advantage in either direction. Whilst we did not employ a 'non inferiority' approach, our results, we contend, are more than adequate to suggest equality given the absence of any identified availability bias, the central tendency of outcome (exact equality)and the moderate degree of potential variation indicated by calculated confidence intervals (and the understanding that the likelihood of variation decreases as we move away from the central tendency).
We were, I believe, adaquately clear about the treatment of multiple-arm trials (for example, collapsing agomelatine arms by combining results of different dose arms).
Finally, reference to older, smaller meta-analyses in this area serves no real purpose as our analysis included all the studies previously included in these analyses and added several more, arguably rendering redundant previous meta-analyses.
Competing interests: I have received fees for lecturing at Servier-sponsored meetings. No funding was received in respect to the study in question.
Taylor et al (2014) have concluded that agomelatine is moderately more effective than placebo and has similar efficacy to standard antidepressants. However there are few points that need to be considered.
Suggestion that agomelatine is significantly more effective than placebo should have been further qualified in statistical and/or clinical terms as statistically significant (p values) results do not necessarily translate into clinical superiority based on generally perceived criteria (for effect size) including the one advocated by the NICE for anti-depressant medications (NICE, 2004).
Interpretation that agomelatine is equally effective as other antidepressants seems to be based on less than adequate interpretation of statistics. Statistically, lack of difference cannot be equated to equivalence of efficacy. There are statistical methods to test equivalence of efficacy parameters and that should have been used to support this specific claim.
It is not clear what corrective measures were put in place to deal with multi-arms trials (if there were any) to manage double/repeat counting of samples from multi-arm trials. Statements like ‘agomelatine is moderately more effective than placebo’ is also unlikely to pass statistical scrutiny. In fact, meta-analyses on this subject (Singh et al, 2012; Koesters et al, 2013; Kasper et al, 2013 and Guaiana et al, 2013) all have come with very similar results, interpretations and recommendations in contrast to the impression from the article by Taylor et al (2014) about a small effect size which happens to be statistically significant. Though, the authors (Taylor et al, 2014) do hold common views regarding its clinical relevance as they have explicitly mentioned in their discussion section, somehow it has not been reflected in the abstract.
• Guaiana G, Gupta S, Chiodo D, Davies SJ, Haederle K, Koesters M. Agomelatine versus other antidepressive agents for major depression. Cochrane Database Syst Rev. 2013 Dec 17;12:CD008851.
• Kasper S, Corruble E, Hale A, Lemoine P, Montgomery SA, Quera-Salva MA. Antidepressant efficacy of agomelatine versus SSRI/SNRI: results from a pooled analysis of head-to-head studies without a placebo control. Int Clin Psychopharmacol. 2013 Jan;28(1):12-9.
• Koesters M, Guaiana G, Cipriani A, Becker T, Barbui C. Agomelatine efficacy and acceptability revisited: systematic review and meta-analysis of published and unpublished randomised trials. Br J Psychiatry. 2013 Sep;203(3):179-87.
• NICE (2004). National Institute for Health and Clinical Excellence. NICE Guidance (http://www.nice.org.uk/nicemedia/pdf/cg023fullguideline.pdf).
• Singh SP, Singh V, Kar N. Efficacy of agomelatine in major depressive disorder: meta-analysis and appraisal. Int J Neuropsychopharmacol. 2012 Apr;15(3):417-28
• Taylor D, Sparshatt A, Varma S, Olofinjana O. Antidepressant efficacy of agomelatine: meta-analysis of published and unpublished studies. BMJ. 2014 Mar 19;348:g1888.
Competing interests: No competing interests
Dr Koesters and colleagues make some important observations on our meta-analysis1 which deserve detailed response.
We included in our review all completed trials of which we were aware and which met our inclusion criteria. We worked with the manufacturer, Servier, to obtain all data from completed acute trials. The study by Montgomery and colleagues2 was excluded because we could not obtain baseline scores for depression rating scales for all subjects and could thus not discern extent of change. The PRISMA flow chart lists this study as “unable to extract data on acute treatment”.1 The small difference in numbers attaining sustained remission in this study suggests some superiority for paroxetine but differences in symptom score change cannot be inferred.
The CL3-0743 study was excluded because Servier were, at the time of communication, unable to supply the dataset for this study (in PRISMA chart – “Dataset not yet available”1). We understand that this was because, although the study itself was complete, data analysis was not. We did not include studies CL3-0274 and CL3-0625 because we were not aware of them. CL3-0274 was a study of hospitalised subjects with treatment resistant depression which would not have met our inclusion criteria (i.e. non refractory subjects). CL3-0625 was a long-term study with a primary endpoint (response rate) at 6 months which Servier considered outside the scope of our request for information. We do not know whether or not acute data are available for this study. It is worth noting that, despite our not including these potentially useful studies, our funnel plots did not indicate any availability bias in respect to included studies.
We agree that changes in liver function tests are somewhat more common with agomelatine than with placebo. Regulatory fears about liver toxicity have provoked mandatory liver function monitoring for agomelatine. As Dr Koesters and colleagues point out, the risk of liver injury (as opposed to liver test function test changes) is unknown. Liver toxicity (sometimes fatal) has, of course, been seen with other antidepressants6 and changes in liver function tests are more common with, for example, MAOIs and amitriptyline than with agomelatine.7
Sub-analysis of comparator trials by drug was undertaken at the behest of the journal’s referees: it was not part of our research plan. We do not think it prudent to estimate effect size using data on sustained remission because this outcome’s relationship to remission, response and symptoms score change (our outcomes) is unknown.
Dr Koesters and co-workers rightly point out that our meta-analysis might not include all relevant studies and we acknowledge this possibility. Nonetheless, our analysis includes significantly more trials and trial subjects than any previous report and so represents the current best estimate of agomelatine’s efficacy and tolerability.
(1) Taylor D, Sparshatt A, Varma S, Olofinjana O. Antidepressant efficacy of agomelatine: meta-analysis of published and unpublished studies. BMJ 2014; 348:g1888.
(2) Montgomery SA, Kennedy SH, Burrows GD, Lejoyeux M, Hindmarch I. Absence of discontinuation symptoms with agomelatine and occurrence of discontinuation symptoms with paroxetine: a randomized, double-blind, placebo-controlled discontinuation study. Int Clin Psychopharmacol 2004; 19(5):271-280.
(3) Vahia V, Yadav A. Efficacy and safety of agomelatine with flexible dose (25 mg/day with blinded potential adjustment at 50 mg) given orally for 8 weeks in Indian outpatients with Major Depressive Disorder A randomised double-blind national multicentric study with parallel groups, versus sertraline (50 mg/day with blinded potential adjustment at 100 mg). CL3-20098-074. 2011.
(4) European Medicines Agency. Refusal CHMP Assessment report for Valdoxan [Procedure No. EMEA/H/C/656, Doc. Ref. EMEA/CHMP/87018/2006]. 2006. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Public_asses....
(5) European Medicines Agency. CHMP Assessment report for Valdoxan [Procedure No. EMEA/H/C/000915, Doc. Ref. EMEA/655251/2008]. 2009. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Public_asses....
(6) Park SH, Ishino R. Liver injury associated with antidepressants. Curr Drug Saf 2013; 8(3):207-223.
(7) Voican CS, Corruble E, Naveau S, Perlemuter G. Antidepressant-induced liver injury: a review for clinicians. Am J Psychiatry 2014; 171(4):404-415.
Competing interests: Professor Taylor has in the past received fees for independent lectures sponsored by Servier, the manufacturer of agomelatine. No funding was received in relation to the current meta-analysis.
The recent review by Taylor and colleagues (1) states: “This study is perhaps a unique example of where working with regulatory authorities and drug manufacturers allows a full and realistic appraisal of the efficacy of a marketed drug” (page 5), and “we can be reasonably confident that we have captured all available data” (page 5). We acknowledge and very much appreciate that Servier, the manufaturer of agomelatine, changed their position and made manufacuterer data avaible to other researchers. We requested the same data with no success in the past.
The authors report that “Servier also confirmed that there were no other completed studies known to them other than those we had ourselves identified and those about which they had notified us.” However, the authors excluded a published study (2) and another unpublished study (3) without giving resaons for the exclusion of these studies. The study by Montgomery and colleagues (2) is a study that examined withdrawal symptoms, but the first phase of the trial is a 12 week double blind randomised comparison of agomelatine and paroxetine. Data from the first study period is not reported in the publication but the number of “sustained responders” randomised to placebo in the second phase of the study is in favour of paroxetine. Considering the authors had access to manufacturer data, they might have been aware of reasons for the exclusion of data from the first phase of this study but if so these were not reported. The other unpublished study excluded (CL3-0743, referred to as 17 in the review) was, according to trial registration data (3), an 8 week randomised, double blind trial comparing agomelatine and sertraline. From the information presented in the trial registry the reason for study exclusion is not evident. We note that the present systematic review seems not to be based on a protocol (there is no reference to a protocol), and no list of excluded studies has been published. Therefore, it is impossible to judge whether the above mentioned trials were not identified or were excluded (and for what reasons).
In our reviews (4,5) we identified two further unpublished studies (CL3-027 and CL3-062) not mentioned in this review. EMA or trial registry information on these unpublished trials is sparse. The study CL3-027 appears to be a 6 month placebo controlled trial in treatment resistant depression6, and CL3-062 was a 6 month double blind controlled trial comparing agomelatine to an SNRI (7). Both studies were long-term studies, but it appears unlikely that these studies did not provide useful data on the acute treatment phase. However, in the absence of a protocol of the systematic review by Taylor and colleagues (1) and in the absence of documentation on the studies excluded it remains unclear whether studies were not available to the researchers or excluded by the authors, and why.
We are also concerned that the statement with respect to hepatic reactions could be misinterpreted. While acknowledging that hepatic reaction was not an objective of their review, Taylor and colleagues report that: “... only three cases of apparent liver toxicity (…) have been reported in the UK since its introduction“, and they concluded that “In neither case do available data allow the calculation of the likely incidence of serious hepatic reaction to agomelatine.” According to the EMA assessment report (7) there seems to be a ”consistent trend throughout the dataset of more cases with potentially clinically important elevation of aminotransferases (> 3 x ULN) among those given agomelatine vs placebo”. A recent assessment report variation from EMA (8) concludes: “The serious hepatotoxic reactions presented in these reports caused concern” (page 45), and it states: “In post-marketing surveillance, 746 cases of Liver adverse reactions have been received since Market Authorisation, including 12 cases of severe hepatic dysfunction.” (page 47). Our analyses (4) showed that there was a threefold risk of abnormal liver values in patients treated with agomelatine compared to patients treated with SSRIs although this data was only reported in very few studies and did not reach statistical significance. We therefore agree that the data is still too sparse to allow definitive conclusions, but we think it should be stressed that an increased risk cannot be excluded. Therefore, careful monitoring is necessary.
In their comparison of agomelatine with other antidepressants, Taylor et al. (1) combined all SSRI and SNRI in one group, although grouping drugs with different mechanisms and different efficacy and safety profiles is questionable. This may have also contributed to statistical heterogeneity and may have masked potential differences. For example, including the study by Montgomery and colleagues (2) by estimating an effect size from the sustained remitters, the comparison of agomelatine and paroxetine nearly reaches statistical significance in favour of paroxetine (see figure). The absolute difference of SMD=-0.15 is small, but it is comparable to the difference between agomelatine and placebo. This means that a minor difference in data analysis may profoundly influence the overall perspective in data interpretation. A published protocol would have allowed readers to understand if the choices made by the review authors followed a priori planning.
1 Taylor D, Sparshatt A, Varma S, Olofinjana O. Antidepressant efficacy of agomelatine: meta-analysis of published and unpublished studies. BMJ 2014;348: g1888.
2 Montgomery SA, Kennedy SH, Burrows GD, Lejoyeux M, Hindmarch I. Absence of discontinuation symptoms with agomelatine and occurrence of discontinuation symptoms with paroxetine: a randomized, double-blind, placebo-controlled discontinuation study. International Clinical Psychopharmacology 2004;19: 271-80.
3 Vahia V. Efficacy and safety of agomelatine with flexible dose (25 mg/day with blinded adjustment at 50 mg) given orally for 8 weeks in Indian outpatients with Major Depressive Disorder A randomised double-blind national multicentric study with parallel groups, versus sertraline (50 mg/day with blinded potential adjustment at 100 mg). EU Clinical Trials Register [www.clinicaltrialsregister.eu] 2010.
4 Guaiana G, Gupta S, Chiodo D, Davies SJC, Haederle K, Koesters M. Agomelatine versus other antidepressive agents for major depression. Cochrane Database Syst Rev 2013.
5 Koesters M, Guaiana G, Cipriani A, Becker T, Barbui C. Agomelatine efficacy and acceptability revisited: systematic review and meta-analysis of published and unpublished randomised trials. The British Journal of Psychiatry 2013;203: 179–87.
6 European Medicines Agency. Refusal CHMP Assessment report for Valdoxan [Procedure No. EMEA/H/C/656, Doc. Ref. EMEA/CHMP/87018/2006], 2006. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Public_asses... (accessed 9 Apr 2014).
7 European Medicines Agency. CHMP Assessment report for Valdoxan [Procedure No. EMEA/H/C/000915, Doc. Ref. EMEA/655251/2008], 2009. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Public_asses... (accessed 9 Apr 2014).
8 European Medicines Agency. Assessment report for a variation including a contraindication, 2013. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Assessment_R... (accessed 11 Apr 2014).
Competing interests: A systematic review on agomelatine efficacy carried out by the authors of this letter (MK, GG, TB, CB) was rejected by the BMJ in 2013. TB reports research funding (unrestricted research grants) to the Department of Psychiatry II, Ulm University, for clinical trials and investigator-initiated trials from AOK Bundesverband (Berlin), AstraZeneca (Wedel), DGPPN (Deutsche Gesellschaft für Psychiatrie, Psychotherapie und Nervenheilkunde, Berlin), and Robert-Bosch-Stiftung (Stuttgart). The department has also received funds to a minor extent for symposia and in-house training from Affectis (Martinsried), AstraZeneca (Wedel), Bristol-Myers Squibb (München), Euroimmun AG, Fresenius Kabi (Bad Homburg), Janssen Cilag (Neuss), Lilly (Bad Homburg), Lundbeck (Hamburg), PreventPharma GmbH, Servier (München). All other authors declare that they have no conflicts of interest.