Balancing benefits and harms: the example of non-steroidal anti-inflammatory drugsBMJ 2004; 329 doi: http://dx.doi.org/10.1136/bmj.329.7456.31 (Published 01 July 2004) Cite this as: BMJ 2004;329:31
- Paul Dieppe, professor1 (, )
- Christopher Bartlett, research assistant1,
- Peter Davey, professor2,
- Lesley Doyal, professor3,
- Shah Ebrahim, professor1
- 1 Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR
- 2 Medicines Monitoring Unit (MEMO), Divisions of Community Health Sciences and Medicine and Therapeutics, University of Dundee, Dundee
- 3 School for Policy Studies, University of Bristol
- Correspondence to: P Dieppe
- Accepted 5 June 2004
To provide safe and effective interventions for people, reliable and valid evidence is needed. This is most easily produced by undertaking trials in samples of people who are as homogeneous as possible and applying the results to similar, well defined groups of patients. To be equitable, however, appropriate care needs to be provided for everyone in the diverse community using health services. Therefore, there is a tension between obtaining scientific evidence that is reliable but which can be applied only to a small subset of the population, and distributive justice that requires that all in need are treated equally appropriately.
Drugs have potential harms as well as benefits. Doctors would like to be able to balance any risks against benefits to derive a therapeutic ratio for each patient, but this is difficult. Formal trials can tell a lot about the efficacy of a drug in a specific context, but unless they are huge and pragmatic they tell less about a drug's toxicity. Post-marketing surveillance may uncover more information on toxicity, but the data usually lack sufficient detail to lead to an understanding of the determinants of adverse reactions. Furthermore, extrapolation of the efficacy or toxicity of a drug in one disease or group of patients to those associated with different diseases or groups can be difficult and misleading.
We examined aspects of these problems in the context of one commonly prescribed class of drugs—non-selective, non-steroidal anti-inflammatory agents (NSAIDs)—and their use in the management of joint pain.
Risks and benefits of NSAIDs in treating joint pain
We compared the types of patients in whom trials of non-selective NSAIDs are conducted with those who receive the drugs in practice. Then we examined the prevalence and associations of adverse events in these two groups.
As NSAIDs are primarily used to treat arthritic pain, we studied trials of these drugs in patients with osteoarthritis, the commonest form of arthritis in the community.1 We collected a comprehensive database of such trials (described elsewhere2) and examined the inclusion and exclusion criteria to determine the sort of people the trial data were derived from. We also looked at the reporting of adverse events. To compare this with the utilisation and toxicity of NSAIDs in the community, we explored the medicines monitoring database in Scotland (www.dundee.ac.uk/memo).3 This large database links all prescriptions in the area to hospital admission records and data on disease, which helps to determine who is being given the drugs and what adverse events have been experienced.
High quality scientific evidence from drug trials may not be generalisable to everyone likely to take the drug
Minority groups and older people are often excluded from trials
Drugs tend to be used for a wider range of indications than those for which they are trialed
People at risk of adverse events are often deliberately excluded from trials
Benefits and harms of drugs are not measured on comparable scales
Large databases linking prescribing to hospital data and other health records are needed to assess the relative benefits and harms of drugs
From the comprehensive dataset of trials on NSAIDs in osteoarthritis,2 we identified 219 eligible papers describing randomised trials for the treatment of osteoarthritis with a non-selective NSAID in at least one arm. These papers were stratified according to the number of patients in each arm (< 100, 100-199, ≥ 200). A random sample of 11% of each of the three strata was selected, by using a list of random numbers, to produce a representative sample of 25 trials for detailed study (14 trials had < 100 patients in each arm, five trials had 100-199, and six trials had ≥ 200). From these papers we extracted the inclusion and exclusion criteria and adverse events. The trials were relatively small (median 67 patients) and brief (mostly ≤ 6 weeks). Overall, 68.5% of the participants were women, aged 62 (SD 5) years; those over 75 were excluded from most trials. Ethnicity was not generally reported. The participants were mainly patients known to have benefited from NSAIDs and in whom the risk of adverse events was small. Thus those with toxicity to NSAIDs or at risk of gastrointestinal or renal problems were specifically excluded. Dropout rates due to gastrointestinal problems such as dyspepsia were reported in about two thirds of the trials, but serious gastrointestinal events such as bleeding were poorly reported. Other serious adverse events (including renal toxicity) were not mentioned in any trial, and the reporting of outcomes was seldom related to age, sex, or ethnicity.
The medicines monitoring database contains a cohort of 131410 patients who were prescribed non-selective NSAIDs between 1989 and 1996.3 Over half (58%) of the cohort was female. Their mean age was 49.7 (range −10 to ≥ 90), and 19013 (14.5%) were aged over 75. From the number of prescriptions written for individual patients, we examined the use of NSAIDs and related that to hospital admission for gastrointestinal or renal problems. Prescribing was common in all age groups, including those over 75, who were excluded from the trials, and in people at high risk of gastrointestinal or renal problems. The risk of hospital admission for gastrointestinal or renal events increased with increasing age, as well as with increasing use of NSAIDs (table). We calculated the events rate in high risk patients (elderly patients and those with coexisting gastrointestinal or renal disease) and low risk patients (those included in the trials). The relative risk of admission for an acute gastrointestinal event was about four times greater in high risk patients and for a renal event about double.
Validity of trials
Much attention is given to the internal validity of trials (whether the data are “true”) and less to the external validity of trial data (whether the data on drugs are generalisable to the population to whom they are to be prescribed). Our study concerned the external validity of trials of conventional NSAIDs. The basic issue we addressed was diversity—that is, the likelihood of varied effectiveness or adverse events of NSAIDs in different patients.
From our examination of the literature on diversity (including age, sex, gender, and ethnicity) we conclude that there is a complex overlap between the many different factors that might lead to social exclusion and to exclusion from either trials or equitable health care.4 For example, older people are often excluded from trials (the rationale often being that they are more vulnerable to comorbidities, adverse drug reactions, and problems with consenting), and older women are noticeably under-represented in trials of many drugs, despite their larger numbers in most developed societies.5 6 Similarly, ethnic minorities are often excluded (sometimes on the basis of language), and there is a clear overlap between ethnic minorities, economic disadvantage, and social exclusion.7
Drugs tested in one condition are often prescribed to people with another condition. We examined trials of NSAIDs in the management of osteoarthritis because that is the commonest cause of joint pain in the community and the main condition for which these drugs are marketed and used.1 8 Although we found that some aspects of the reporting of these trials was poor, the quality of the trials was generally good, and most of them were carried out on patients with well characterised osteoarthritis, in whom comorbidities were absent (by exclusion). In the population cohort that we studied the situation was different. It is likely that many of those who were prescribed an NSAID had osteoarthritis, but many will have had other causes of pain, including inflammatory forms of arthritis such as rheumatoid arthritis or systemic lupus erythematosus. Similarly, many of those patients will have had comorbidities. Unfortunately, we do not know why the drugs were prescribed to people in the cohort or the level of comorbidity and coprescribing. As a result we cannot make inferences about the mechanisms of harms; although all the patients used NSAIDs, we cannot attribute all the events to NSAIDs. What is clear is that the harms observed in practice were different from those reported in the trials, on which the use of the drugs was based, and that there was a dose relation between risk and drug use.
Pharmaceutical companies promote NSAIDs by highlighting both the benefits and the harms from their use. For example, an advertisement may contain an image of an older person enjoying pain free activity that is contrasted with the image of an ulcerated stomach, if protection of the gastrointestinal tract is the marketing angle.
Adverse event reporting
The recent literature on adverse events associated with NSAIDs has focused on gastrointestinal problems, with less attention to renal and cardiovascular toxicity (fluid retention, hypertension, impairment of renal function), despite these well known problems.9–13 Some of the early literature gave conflicting messages about the association between NSAIDs and renal disease14; but our results are consistent with four large case-control studies from the past 15 years, in which patients, largely elderly, admitted to hospital with renal impairment were 2-4 times more likely to have used NSAIDs than control patients.15–18
Exclusion from randomised controlled trials of patients at risk from renal impairment misinforms practitioners in two important ways: by weakening their awareness of the risk of renal failure associated with NSAID and by failing to provide relevant information about the relative risks from different NSAIDs. Interest in gastrointestinal problems may be partly due to the pharmaceutical industries' endeavours to produce selective NSAIDs, or “coxibs,” that might be less toxic to the gastrointestinal tract.19–21 We have studied only the non-selective NSAIDs, but we have shown that admissions to hospital for renal failure and gastrointestinal events were increased in those using the drugs, particularly older people and those with known risk factors.
Risk benefit ratio
These findings illustrate some of the difficulties in trying to balance the potential risks of an intervention with its potential benefits. Data from trials on efficacy can provide estimates of the absolute effects of treatment by numbers needed to treat or numbers needed to harm. But these treatment effects are averages derived from highly selected, relatively small, and restricted groups of patients who are unlikely to be representative of the population that will use the drugs. A further concern is that numbers needed to treat and numbers needed to harm are often not measured on comparable scales of benefit and harm. If a drug clearly has major life saving benefits, then it is likely that these will outweigh relatively uncommon adverse effects. In the case of NSAIDs, however, comparing improvements in joint pain and stiffness with the uncommon risks of hospital admission for gastrointestinal bleeding or renal failure is not straightforward. Similarly, it is clear that assuming that the risk of harm is relatively stable whereas that of benefit depends on the severity of the disease, cannot be applied in this sort of case, due to the likely effects of disease indication, age, and comorbidity on the adverse events.
We looked specifically at trials of NSAIDs in osteoarthritis. Many trials have also been done in rheumatoid arthritis, but relatively few in other, less common rheumatic diseases such as systemic lupus erythematosus. Theoretically, renal adverse events could be greater in patients with rheumatoid arthritis, and particularly those with systemic lupus erythematosus, than in those with osteoarthritis, as these and other systemic rheumatic conditions can affect the kidneys and cardiovascular systems directly.22 23 One problem with most of the less common inflammatory rheumatic diseases, in which there are few trials, is that NSAIDs are not specifically approved for use in these conditions but are nevertheless prescribed routinely to patients with them if they have joint pain, and there is no way of knowing from the trial data whether that is damaging to the kidneys or cardiovascular system. Similarly, both gastrointestinal and renal adverse events are affected by coprescribing, including steroids and analgesics, both of which are commonly used for arthritis.24 Again, the importance of coprescribing cannot be certain from the trial data alone.
The need for record linkage
Our study was only possible because of access to the Medicine Monitoring Unit's database, in which prescribing is linked to hospital data through collaboration with NHS Information Services (ISD) in Edinburgh. The use of this sort of record linkage is particularly valuable for the examination of adverse events, but if reliable information is to be obtained, large datasets that can link prescribing data to other health records are needed. In many countries, record linkage has been made more difficult with the introduction of data protection legislation.25 26 The best way to get more reliable post-marketing data on drug toxicity in particular is by establishing more datasets like the one in Tayside, so that adverse events can be related to patient characteristics, and by performing large randomised trials using routine data for follow up.27
We thank Jiri Chard for access to the trials' database, Tom MacDonald for his contributions to the work on the Medicines Monitoring Unit database, and Steve Morant for analysis of the database.
Funding This study was funded by the NHS Research and Development Programme and will be reported in full in an Health Technology Assessment report. Additional funding was supplied by the Medical Research Council's Health Services Research Collaboration. Contributors: PDi initiated the study and was coprincipal investigator. He will act as guarantor for the paper. CB carried out the review and analysis of NSAID trials. PDa was responsible for the work on the Medicines Monitoring Unit database. LD led the work on diversity and exclusions of trials. SE was coprincipal investigator.
Competing interests PDa has been reimbursed for attending conferences and had research projects funded by Aventis; Bayer; GlaxoSmithKline; Merck, Sharp, and Dohme; and Pfizer. These were not in relation to non-steroidal anti-inflammatory drugs or related products.