What can we learn from observational studies of oseltamivir to treat influenza in healthy adults?BMJ 2009; 339 doi: http://dx.doi.org/10.1136/bmj.b5248 (Published 08 December 2009) Cite this as: BMJ 2009;339:b5248
- Nick Freemantle, professor of clinical epidemiology and biostatistics,
- Mel Calvert, senior lecturer
- Correspondence to: N Freemantle
The treatment effects of drugs are conventionally estimated in randomised controlled trials. Random allocation of patients ensures that any difference in outcome observed between the experimental groups must be attributable either to chance or to the randomised treatments. Although a remarkably durable approach, randomisation does have limitations. Randomised trials conducted for regulatory purposes rarely include people with comorbidities, since adverse events observed as a result of these comorbidities may undermine the attempts to establish the safety of the new drug. Furthermore, randomised trials are costly and time consuming and may not be conducted by industry sponsors when a “positive” outcome seems unlikely. Thus health policy makers often have to make important decisions about new drugs when not all relevant trials have been undertaken. This is the situation currently regarding the use of antiviral therapy in H1N1 influenza. It may be argued that at such times we should be informed by all available evidence and not constrained by randomised trials alone.
The authors of the Cochrane Collaboration systematic review of neuraminidase inhibitors for preventing and treating influenza in healthy adults1 approached Roche, the manufacturer of oseltamivir, for unpublished data. Among the incomplete data that Roche provided was a list of observational studies that it considered to describe the “real life” value of antiviral treatment. The BMJ invited us to provide a rapid review of these studies and the extent to which they support the use of oseltamivir to treat influenza in healthy adults.
Problems of non-randomised trials
In non-interventional study designs the difference in outcomes between those receiving and not receiving the treatment of interest may be due to the play of chance, to the treatment, or to some additional confounder. Excluding chance as an explanation for an observed difference in outcome thus leaves two rather than one possible explanations, with obvious consequences for our certainty about the results. These biases essentially relate to some kind of preferential selection of patients to different treatment options. It is particularly challenging when the characteristics of a patient are likely to inform decisions to use a drug. Thus clinical judgment may mean that a patient at raised risk of a poor outcome is more likely to be prescribed a particular drug. If this increased risk is incompletely described by risk stratification methods, the treatment may appear relatively harmful,2 a troublesome bias that is referred to as confounding by indication.3
Thus, when appraising observational evidence, in addition to considering the study design and likely generalisability of findings, the reader must also consider the appropriateness of attempts by the authors to overcome potential confounding and the likely size and direction of any residual confounding.
Assessment of studies
The nine post-marketing studies identified by Roche included 529 122 patients, of whom 190 817 were prescribed oseltamivir during 1999 to 2007 (table⇓).4 5 6 7 8 9 10 11 The databases from which patients were identified were inadequately described, making it difficult to ascertain the extent to which different studies may have included the same participants. One study of oseltamivir in people with diabetes was explicitly a subgroup analysis of data included in previous studies.10
The study groups varied and included the general population,4 5 people aged 65 years or younger,6 adults,11 people aged 13 years or older,7 children,8 children with chronic medical conditions,9 adults with diabetes,10 and adults with a history of cardiovascular disease.12 Thus several studies include participants that are outside the focus of Jefferson et al’s systematic review, which focused on healthy adults.1
The main outcome in all but two studies was the development of pneumonia or other direct complications of influenza. One study examined the risk of stroke or transient ischaemic attack.11 A further study examined major cardiovascular events in people with cardiovascular disease.12 The reporting of outcomes was poor in several studies. For example, transient ischaemic attack and strokes were reported together only in Madjid et al.11
It was unclear the extent to which studies were driven by prespecified protocols and analysis plans, although neutral results were reported alongside beneficial results for oseltamivir.
The studies all had specific inclusion criteria, resulting in exclusion of large numbers of people. In particular, the studies did not include people who received antiviral drugs other than oseltamivir or who received oseltamivir more than 1-2 days after diagnosis of influenza.
In some studies, significant differences were observed in the baseline characteristics of the oseltamivir and control groups, suggesting that higher risk patients may have been captured in the control group.4 5 7 8 11 One study did not give a detailed description of participants’ comorbidities (such as pneumonia, chronic obstructive pulmonary disease, asthma, and diabetes) but provided a summary measure of comorbidity burden.6 Another did not record pre-existing respiratory illness.12 One study did not consider comorbidity or diabetes.10 Significant differences in geographical location and prescribing were observed in several studies5 8 9 10 or were not described.11 12 All studies used multivariable models to adjust for known confounders. Several studies used propensity score methods to attempt to deal with confounding by indication (table⇑).
Results of multivariable analyses were broadly in line with those reported in the randomised trials. For participants with clinically diagnosed influenza, the estimated number who needed to be treated with oseltamivir to avoid one diagnosis of pneumonia was always over 100, and may be as high as 1000. Only one of the observational studies reviewed here considered safety issues.4
Although the studies reviewed were of variable quality, they generally support the conclusion that oseltamivir may reduce the incidence of pneumonia and other consequences of influenza in otherwise healthy adults. However, these events are rare, so for most otherwise healthy adults treatment of influenza with oseltamivir is not likely to be clinically important.
A potential advantage of observational studies is that they can provide evidence on the use of a drug in a realistic setting. However, this advantage was largely undermined by the studies’ selection criteria, which tended to exclude people who received oseltamivir later than the recommended time frame. Matched propensity scoring approaches may also undermine the generalisability of results if extreme values are excluded because of inability to achieve a match. We might also expect the effectiveness of oseltamivir to be reduced in the clinical setting, where treatment may be based on clinical criteria alone, although use of different outcome measures in the randomised trials and observational studies prevents us from answering this question empirically.
Several studies suggested that the benefits of oseltamivir may be underestimated if patients delayed seeking care (date of diagnosis may not represent onset of symptoms), or if patients did not take the drug as prescribed. However, this may provide a more realistic view of real life practice, and with increasing drug resistance estimates of effectiveness may now be reduced.
The estimated effect of antiviral drugs in people with existing cardiovascular disease was substantial, and the difference in rates of death and serious morbidity were potentially clinically important.12 However, those receiving oseltamivir were younger and thus at lower risk. As the authors of that study commented, these findings could usefully be examined in further randomised trials.
Interpretation of the studies was difficult. Differences in baseline comorbidity or geographical distribution were present in several studies. It seems likely that some patients were included in more than one study, which undermines the ability of these studies to provide independent estimates. The generalisability of findings from employer sponsored health insurance databases may also be questioned.
The direction of any bias from confounding by indication is uncertain. All the studies were conducted in the United States, where more wealthy subjects, who are likely to have better outcome and to engage in more positive health related behaviours, might be more likely to be prescribed an antiviral drug. However, patients thought likely to be particularly at risk of poor outcome may also be more likely to be prescribed antiviral drugs. The focus on patients who received oseltamivir within a day or so of diagnosis of influenza and comparison with people who had no antiviral drug prescribed, seems likely to exacerbate the effects of such confounding and reduce the extent to which these observational studies might be considered to reflect real world practice. Although several studies made appropriate attempts to address unknown confounders by using propensity scores, these methods do not ensure that confounding by indication is avoided.3
Only one study explicitly examined drug safety.4 This is an important omission given the widespread use of oseltamivir. An earlier study by Enger and colleagues on the Ingenix United Healthcare database, which seems to have been conducted for regulatory purposes, concluded that there was no association between oseltamivir use and an increase in the risk of cardiac or neuropsychiatric events on the basis of confidence intervals that were wholly on the side of benefit on those outcomes for oseltamivir exposure.13 It is unclear why similar results in later studies (often including some of the same authors) were interpreted as providing evidence that oseltamivir reduced the risk of major cardiovascular events or stroke in high risk patients rather than providing evidence of safety, given the intrinsic biases associated with these non-randomised comparisons.
Our rapid review of these “real life” data suggests that oseltamivir may reduce the risk of pneumonia in otherwise healthy people who contract flu. However, the absolute benefit is small, and side effects and safety should also be considered. None of the studies examined the role of oseltamivir in patients with H1N1 influenza, which may be associated with higher rates of pneumonitis than seasonal influenza.14 We did not consider the evidence for the use of oseltamivir in high risk patients, although several of the studies identified by Roche were in special populations. Other observational studies suggest that early intervention with antivirals for influenza may benefit a range of high risk patients and potentially improve survival rates, but these studies are also open to residual confounding.13 14 15 16 17 18 19 20
A potential bias in our review derives from considering only studies identified by the manufacturer. Had we identified evidence that was supportive of the use of oseltamivir in the treatment of otherwise healthy subjects with H1N1 virus it would have been important to search more broadly for any additional studies to avoid publication bias.
Cite this as: BMJ 2009;339:b5248
Contributors: NF and MC were both involved in the conception, design, analysis, and interpretation of data, drafting the article and revising it critically for important intellectual content. Both gave final approval of the version to be published. NF is the guarantor.
Competing interests: The authors have completed the unified competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare (1) the BMJ helped them access three articles not available through their university library; (2) they supervise a PhD student who is supported and employed by Roche; (3) their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and (4) they have no non-financial interests that may be relevant to the submitted work.
Provenance and peer review: Commissioned; externally peer reviewed.