Intended for healthcare professionals

Research News

Study claiming Tamiflu saved lives was based on “flawed” analysis

BMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g2228 (Published 19 March 2014) Cite this as: BMJ 2014;348:g2228

Re: Study claiming Tamiflu saved lives was based on “flawed” analysis

Dr Jones has misunderstood our methods as described in the paper Muthuri et al. (2014) and our response to his previous critique.

Our published paper reported the results of two separate survival analyses using a Cox regression shared frailty model. The first survival analysis was prompted by the question of whether treatment with NAI antivirals was associated with a reduction in mortality as compared to no NAI antiviral treatment. For this analysis we followed the exact approach suggested by Dr Jones, i.e. we modelled NAI treatment as a time-dependent covariate that, ‘equals 0 while the patient is untreated, then becomes 1 when treatment begins’ (with NAI untreated patients being coded as ‘0’ for the entire duration of follow-up). This analysis yielded an adjusted hazard ratio (HR) of 0.51 (95% confidence intervals, CI: 0.45-0.58); p<0.0001.

The survival curves in Figure 2 on the other hand, do not relate to this survival analysis; this second survival analysis was prompted by the question of whether the time to treatment initiation is associated with mortality among NAI antiviral treated patients only. For this second survival analysis the effects associated with time to initiation of treatment were explored by stratifying treatment by categories denoting ‘time to treatment initiation from illness onset’. This analysis showed that there was an incremental increase in the mortality hazard with each day’s delay in initiation of treatment up to day 5 as compared with treatment initiated within 2 days of symptom onset [adjusted HR 1.23 (95% CI: 1.18-1.28)]; p<0.0001.

We fear Dr Jones has again misread and/or misinterpreted our paper when he claims that our 'so-called time-dependent analysis (hazard ratio 0.51) is impossible compared to your analysis where treatment is assumed time-independent (relative risk 0.81)'. The risk estimate he quotes, [adjusted odds ratio (OR): 0.81 (95% CI: 0.70-0.93); p=0.0024] is an adjusted odds ratio obtained from a generalised linear mixed model. A direct comparison of this to the adjusted hazard ratio [adjusted HR: 0.51 (95% CI: 0.45-0.58); p<0.0001] obtained from the Cox regression shared frailty model is not appropriate.

Whilst we agree that the paper by Beyersmann et al. (2008) cited by Dr Jones offers mathematical proof that a time-dependent analysis should diminish the effect size in favour of treatment, this is predicated on the assumption that the only time dependent bias at work is immortal time bias. Other time-dependent biases may artificially increase the hazard associated with treatment for example, selection biases related to illness severity/stage of illness; we have repeatedly made the point in the paper and in our previous response that severely ill patients were frequently diagnosed with influenza, late on in the course of the illness and that antiviral drugs were similarly started late in patients who by that stage may have had little chance of survival. Therefore performing time-dependent analyses will not only remove immortal time bias, it can reduce other forms of time-dependent bias. Thus, the results from our time dependent analysis are entirely realistic and our statistical approach is sound.

We hope we have assured Dr Jones that we have already conducted a proper time-dependent analysis. We accept the limitation of missing data on timing of treatment in our pooled sample. This is precisely why even though the Cox regression model is generally considered superior to logistic regression models (Kleinbaum and Klein, 2012) we opted to use a multilevel logistic regression model as our primary analysis strategy, so as to include all treated patients in the analysis. Even then we observed a statistically significant association between NAI antiviral use (irrespective of stage of illness at which they were administered) and reduced mortality [adjusted OR: 0.81 (95% CI: 0.70-0.93); p=0.0024]. In the case of early NAI antivirals administered within 2 days of illness onset, the association with reduced mortality was even more marked [adjusted OR: 0.50 (95% CI: 0.37 -0.67); p<0.0001].

References

Muthuri SG, Venkatesan S, Myles PR, et al. (2014) Effectiveness of neuraminidase inhibitors in reducing mortality in patients admitted to hospital with infl uenza A H1N1pdm09 virus infection: a meta-analysis of individual participant data. Lancet Respir Med; published online March 19. http://dx.doi.org/10.1016/S2213-2600(14)70041-4

Beyersmann et al. (2008) An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation. Journal of Clinical Epidemiology 61: 1216-1221.

Kleinbaum, D.G. and M. Klein, Survival Analysis: A Self-Learning Text. Third Edition ed. 2012, New York: Springer.

Competing interests: Co-authors of the paper Muthuri et al. (2014) that is being critiqued

02 April 2014
Puja Myles
Public Health Specialist/Epidemiologist
Jo Leonardi-Bee, PRIDE Research Consortium investigators
University of Nottingham
Clinical Sciences Building, City Hospital, Nottingham