Research News

Study claiming Tamiflu saved lives was based on “flawed” analysis

BMJ 2014; 348 doi: (Published 19 March 2014) Cite this as: BMJ 2014;348:g2228

Re: Study claiming Tamiflu saved lives was based on “flawed” analysis

Thank you for your explanation of how you conducted your time-dependent analysis. It is now clear why you have obtained such a biased estimate of treatment effect. By ignoring the time prior to NAI treatment you have increased the immortal time bias, not eliminated it. In fact you have not included NAI treatment as a time-dependent exposure; you have just begun the follow up at different points in time for each treatment group. You have failed to take into account that patients in both treatment groups needed to survive long enough to reach hospital. Once they reached hospital they needed to survive long enough to get the opportunity to receive NAI treatment. The way to eliminate immortal time bias is to begin follow up at the same time in each treatment group at the logical beginning of follow up which is hospital admission. Assuming all patients were untreated prior to admission then all patients begin follow up untreated. Once a patient begins NAI treatment they then join the NAI treatment group. This analysis can easily be conducted using a Cox regression model where NAI treatment is a time-dependent covariate that equals 0 while the patient is untreated, then becomes 1 when treatment begins. Van Walraven, et al state that time-dependent bias is always in favour of treatment and Beyersmann, et al prove this mathematically. This is why your result for your so-called time-dependent analysis (hazard ratio 0.51) is impossible compared to your analysis where treatment is assumed time-independent (relative risk 0.81).

The bias always favours the treatment group because the time from initiation of follow up to initial treatment exposure is incorrectly allocated to the treatment group in an analysis that incorrectly includes treatment exposure as time-independent. This has the effect of artificially inflating the number of patients at risk in the treatment group and conversely artificially deflating the number of patients at risk in the non-treatment group. This makes the estimates of mortality over time too low for the treatment group and too high for the non-treatment group. An analysis that correctly includes treatment exposure as time-dependent therefore diminishes any treatment effect in favour of treatment or even changes its direction.

In light of this I hope that you can conduct a proper time-dependent analysis and report your results. This I believe will provide a much more realistic estimate of the effect of NAI treatment on mortality. However it will not help with the problem of 35% missing data which also introduces bias because the patients with missing timing of treatment did most poorly of all. For the missing data you could use multiple imputation based on a regression model for predicting time from admission to treatment.

1. Van Walraven, et al. Time-dependent bias was common in survival analyses published in leading clinical journals. Journal of Clinical Epidemiology 57 (2004) 672–682.

2. Beyersmann, J., Gastmeier, P., Wolkewitz, M., Schumacher, M. , An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation. Journal of Clinical Epidemiology, 2008. 61: p. 1216-1221.

Competing interests: I am an unfunded researcher working on a Cochrane Review of neuraminidase inhibitors for influenza

28 March 2014
Mark Jones
University of Queensland
Brisbane, Australia