Steroid injections through the eardrum reduce dizziness in Meniere’s disease, study finds
BMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i6185 (Published 18 November 2016) Cite this as: BMJ 2016;355:i6185
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Patel and colleagues [1] report on a double-blind, two-armed, randomized controlled study in patients with unilateral refractory Meniere's disease (MD) comparing treatment effects of two different intratympanic injections: methylprednisolone (experimental) and gentamicin (comparator). The trial is based on evidence that intratympanic gentamicin seems effective compared to placebo at reducing vertigo attack rates in this target population [2,3].
The authors conclude that there is no evidence that the therapies differ with respect to a reduction of attack rates. The primary comparison was the mean difference (experimental minus comparator) in attack rates during the predefined 6-month assessment period (months 19 to 24) resulting in delta = –0.9 (95% CI –3.4 to 1.6), implying that the mean attack rate during the evaluation period is 0.9 (but not significantly) higher in the active control compared to the experimental group. However, no evidence of a difference is not evidence of no difference [4].
In terms of the primary efficacy analysis it would have been more appropriate to use a difference in absolute change approach ("difference in difference") comparing both groups with respect to the mean change in attack rates between baseline (6-month pre-enrollment phase) and end-of-trial (months 19 to 24). A change of 17.4 on active control and of 14.8 on experimental treatment implies a mean difference in absolute change (experimental minus comparator) of –2.6, i.e. the change on active control is 2.6 attacks larger than on the experimental drug. Unfortunately, Patel et al. did not report the standard error for the change from baseline per group, and a corresponding confidence interval cannot be determined. Using a difference in absolute change approach increases the power to discover an effect. A recommended method of analysis would be to adjust for the baseline attack rates [5-7].
Analyzing the attack rates as normally distributed also may have reduced the power of the study. According to table 2, mean and standard deviation of the number of attacks are of comparable size, implying a non-symmetric distribution with a long right tail. Transforming the data to achieve a more symmetric distribution may have resulted in more power (smaller confidence intervals) when an ANOVA or t-test is applied. Assuming a distribution for count response data would have been more appropriate [8].
The sample size calculation of this trial was designed to detect a difference in hearing loss, the most clinically relevant side effect in ablative gentamicin drug treatment and defined as secondary outcome. Thus, sample size calculation is not related to the main study objective (difference in change in attack rates), leaving uncertainties in terms of statistical power to detect a minimum clinically relevant difference in the primary endpoint [9].
The primary efficacy outcome was assessed during face-to-face interviews at the end of two six-month reporting periods. In confirmatory trials on chronic conditions such as MD a blinded, reliable and valid assessment of patient's subjective experience of symptoms together with real-time data collection is crucial to minimize recall bias. Disease-specific, standardized, validated electronic or paper-based, event-oriented or daily symptom diaries that are designed to be continuously completed by patients at home offer a substantial benefit over retrospective data collection based on face-to-face interviews at clinic visits [10,11].
The logic of the paper is that of a non-inferiority trial (which seeks to show the investigational product to be comparable) and does not fit a superiority trial (prove a difference): methylprednisolone is not inferior to gentamicin in terms of reducing the attack rate, but may have the potential to be superior in other (safety) outcomes (in particular hearing loss). A non-inferiority trial needs a non-inferiority margin, i.e., which maximal difference between active control and experimental drug is considered clinically irrelevant [12,13]. For this inferiority trial a per-protocol analysis would be performed to calculate the 95% confidence interval (CI) for the treatment difference - driven by the wish to construct conservative treatment effects. If the 95% CI is above the non-inferiority margin the null hypothesis 'The experimental treatment is inferior to the active treatment' would be rejected. A non-inferiority study might have avoided the aporia from not knowing how to interpret the non-significant trial results in a clinically relevant way. Many of those who read the paper [1] will probably use the non-inferiority approach to interpret the results. Since non-inferiority also implies that the experimental treatment may be slightly worse than the comparator, it is possible that its effect comes close to that of a placebo treatment [14]. Therefore, non-inferiority trials should include a third placebo arm to assess whether the comparator's and experimental's effects are superior to the placebo effect. This may be an issue since the attack reduction in the experimental group is on average 2.6 less compared to the comparator.
REFERENCES:
[1] Patel M, Agarwal K, Arshad Q, et al. Intratympanic methylprednisolone versus gentamicin in patients with unilateral Meniere's disease: a randomised, double-blind, comparative effectiveness trial. Lancet 2016; 388(10061): 2753-62.
[2] Pullens B, van Benthem PP. Intratympanic gentamicin for Meniere's disease or syndrome. Cochrane Database Syst Rev 2011; 3: CD008234.
[3] Phillips JS, Westerberg B. Intratympanic steroids for Meniere's disease or syndrome. Cochrane Database Syst Rev 2011; 7: CD008514.
[4] Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ 1995; 311(7003): 485.
[5] Vickers AJ, Altman DG. Statistics notes: Analysing controlled trials with baseline and follow up measurements. BMJ 2001; 323(7321): 1123-4.
[6] Van Breukelen GJP. ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies. J Clin Epidemiol 2006; 59(9): 920-5.
[7] Egbewale BE, Lewis M, Sim J. Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study. BMC Med Res Methodol 2014; 14: 49.
[8] Adrion C, Fischer CS, Wagner J, Gurkov R, Mansmann U, Strupp M. Efficacy and safety of betahistine treatment in patients with Meniere's disease: primary results of a long term, multicentre, double blind, randomised, placebo controlled, dose defining trial (BEMED trial). BMJ 2016; 352: h6816.
[9] Clark T, Berger U, Mansmann U. Sample size determinations in original research protocols for randomised clinical trials submitted to UK research ethics committees: review. BMJ 2013; 346: f1135.
[10] Gater A, Coon CD, Nelsen LM, Girman C. Unique Challenges in Development, Psychometric Evaluation, and Interpretation of Daily and Event Diaries as Endpoints in Clinical Trials. Ther Innov Regul Sci 2015; 49(6): 813-21.
[11] Dawson J, Doll H, Fitzpatrick R, Jenkinson C, Carr AJ. The routine use of patient reported outcome measures in healthcare settings. BMJ 2010; 340: c186.
[12] Kaul S, Diamond GA. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med 2006; 145(1): 62-9.
[13] FDA Guidance for Industry. Non-Inferiority Clinical Trials to Establish Effectiveness. Food and Drug Administration, U.S. Department of Health and Human Services, Center for Drug Evaluation and Research, November 2016. URL: https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformat...
[14] Pigeot I, Schafer J, Rohmel J, Hauschke D. Assessing non-inferiority of a new treatment in a three-arm clinical trial including a placebo. Stat Med 2003; 22(6): 883-99.
Competing interests: No competing interests
REPLY TO ADRION ET AL. ON PATEL ET AL.
We apologise for not having noticed earlier Adrion and colleagues’ comments on our study comparing the effectiveness of intratympanic Methylprednisolone to Gentamicin in refractory Ménière’s disease [1], covered by the BMJ [2]. Contrary to our expectations, both drugs were equally effective in controlling vertigo (primary outcome).
Adrion’s main concerns were, (i) that negative studies pose the problem of “not knowing how to interpret the non-significant results in a clinically relevant way” and, (ii) the lack of a power calculation for the primary outcome.
Regarding (i), our interpretation is that given the known ototoxic effects of gentamicin, ascertaining any minuscule difference between the two drugs is not justified and, very likely, unethical. A further comment was that “transforming the data to achieve a more symmetric distribution may have resulted in more power when an ANOVA or t-test is applied”. We agree that some of the data deviated from strict normality. There is always debate about the robustness of ANOVA in these cases, however this was addressed in the paper - non-parametric tests confirmed that there were no significant differences between Methylprednisolone and Gentamicin for vertigo outcomes ([1] Appendix, page 6).
Adrion also recommended adjustment for baseline differences, but a crucial strength of our study was adherence to the CONSORT 2010 guidelines for transparent reporting of trials [3], which states that “Although proper random assignment prevents selection bias, it does not guarantee that the groups are equivalent at baseline. Any differences in baseline characteristics are, however, the result of chance rather than bias. The study groups should be compared at baseline for important demographic and clinical characteristics so that readers can assess how similar they were…….Adjustment for variables because they differ significantly at baseline is likely to bias the estimated treatment effect.” Nevertheless, to satisfy Adrion and colleagues’ concerns and for the benefit of readers, we reanalysed our primary outcome data with ANCOVA using baseline vertigo attack frequency over 6 months as covariate. As with ANOVAs and non-parametric tests, the re-analysis showed no difference between drugs (F=.527 df=1,57 p=.471). In other words, performing the ANCOVA does not change our results nor add any further information for the reader.
Regarding (ii) the reason for the lack of power calculation for the primary outcome was clearly stated in our paper: “At the time of planning of our study, there were few published data on intratympanic gentamicin versus steroid treatment for Ménière’s disease”. Hence sample-size calculations were based on hearing outcomes where drug differences were better established [4]. However, we are now able to back-calculate our data. Depending on which outcome metric is chosen to resolve the small effects observed (vertigo attacks at 2 years, percentage reduction in vertigo attacks, absolute reduction in number of vertigo attacks, using intention to treat or per protocol), for conventional 80% power with p=.05 2-tailed the numbers of patients required is between 150 to over 400 per group, thus illustrating the minute differences, if any, for the primary outcome (vertigo).
Finally, the comment “the primary endpoint may be biased since it was assessed by retrospective face-to-face interviews instead of event-oriented or daily symptom diaries”, was also addressed in the paper. It would be unethical to recruit patients with refractory Meniere’s disease experiencing more than one attack per week (Table 1,[1]) and require that they fill up a diary for the next 6 months before receiving treatment. Further, all the validated vestibular questionnaires used showed no differences between Methylprednisolone and Gentamicin either. In the same vein, it would be unethical to give patients a placebo when gentamicin is more effective than placebo for vertigo control (for Cochrane Review, see [5]) and it should be emphasised that a comparison between Gentamicin and Placebo led to the early termination of a randomised, double-blind, controlled trial in 2014 [6]. In view of our data [1], we believe that the initial treatment choice between an ototoxic drug (Gentamicin) and one that it is not (Methylprednisolone), will be simplified for most patients and doctors.
References
[1] Patel M, Agarwal K, Arshad Q et al. Intratympanic methylprednisolone versus gentamicin in patients with unilateral Meniere's disease: a randomised, double-blind, comparative effectiveness trial. Lancet 2016; 388(10061): 2753-62.
[2]. Mayor S. Steroid injections through the eardrum reduce dizziness in Meniere’s disease, study finds. BMJ 2016; 355:i6185.
[3] Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c332.
[4] Sennaroglu L, Sennaroglu G, Gursel B et al. Intratympanic dexamethasone, intratympanic gentamicin, and endolymphatic sac surgery for intractable vertigo in Meniere's disease. Otolaryngology--head and neck surgery: official journal of American Academy of Otolaryngology-Head and Neck Surgery 2001; 125(5): 537-43.
[5] Pullens B, van Benthem PP. Intratympanic gentamicin for Ménière's disease or syndrome. Cochrane Database Syst Rev. 2011;(3):CD008234.
[6] Bremer HG, van Rooy I, Pullens B, et al. Intratympanic gentamicin treatment for Ménière's disease: a randomized, double-blind, placebo-controlled trial on dose efficacy - results of a prematurely ended study. Trials 2014;15:328.
Competing Interests: None
Competing interests: No competing interests