Computer support for determining drug dose: systematic review and meta-analysisBMJ 1999; 318 doi: https://doi.org/10.1136/bmj.318.7189.984 (Published 10 April 1999) Cite this as: BMJ 1999;318:984
- Robert Walton (), research fellowa,
- Susan Dovey, visiting research fellowa,
- Emma Harvey, research fellowb,
- Nick Freemantle, senior research fellow..c
- aICRF General Practice Research Group, University of Oxford, Department of Public Health and Primary Care, Institute of Health Sciences, Oxford OX3 7LF
- bDepartment of Health Sciences and Clinical Evaluation, Alcuin College, York YO1 5DD
- cCentre for Health Economics, University of York, York YO1 5DD
- Correspondence to: Dr Walton
- Accepted 9 November 1998
Objective: To review the effectiveness of computer support for determining optimum drug dose.
Design: Systematic review of comparative studies where computers gave advice to clinicians on the most appropriate drug dose. Search methods used were standard for the Cochrane Collaboration on Effective Professional Practice.
Subjects: Comparative studies conducted worldwide and published between 1966 and 1996.
Main outcome measures: For qualitative review, relative percentage differences were calculated to compare effects of computer support in different settings. For quantitative data, effect sizes were calculated and combined in meta-analyses.
Results: Eighteen studies met the inclusion criteria. The drugs studied were theophylline, warfarin, heparin, aminoglycosides, nitroprusside, lignocaine, oxytocin, fentanyl, and midazolam. The computer programs used individualised pharmacokinetic models to calculate the most appropriate dose. Meta-analysis of data from 671 patients showed higher blood concentrations of drug with computer support (effect size 0.69, 95% confidence interval 0.36 to 1.02) and reduced time to achieve therapeutic control (0.44, 0.17 to 0.71). The total dose of drug used was unchanged, and there were fewer unwanted effects of treatment. Five of six studies measuring outcomes of care showed benefit from computer assistance.
Conclusions: This review suggests that using computers to determine the correct dose of certain drugs in acute hospital settings is beneficial. Computers may give doctors the confidence to use higher doses when necessary, adjusting the drug dose more accurately to individual patients. Further research is necessary to evaluate the benefits in general use.
This systematic review of studies examining computer support for determining optimum drug dose showed benefits from computer use
Computer support led to patients having increased blood concentrations of drug, reduced time to achieve therapeutic benefits, and fewer unwanted effects of treatment
Computer support helps doctors to tailor drug doses more closely to the needs of individual patients
All the studies took place in hospitals, and further research is needed to determine the risks and benefits of widespread use of computer support, particularly in general practice, where most prescribing takes place
Maintaining therapeutic drug concentrations is a complex task requiring knowledge of medicine and pharmacokinetics, a good rapport with the patients, and some skill in calculating dose. Harm can be caused by miscalculating doses because many drugs have a narrow “window” in which therapeutic benefits can be obtained at a low risk of unwanted effects.
Monitoring drug treatment to optimise effects and minimise dangers can be time consuming and requires meticulous attention to detail. Doctors sometimes make errors of judgment because their capacity to process information is exceeded,1 and their computational skills may be inadequate to perform calculations about drug dose.2 For example, 82 of 150 hospital doctors were unable to calculate how many milligrams of lignocaine were in a 10 ml ampoule of 1% solution.3
Computers, however, are very good at gathering information and performing repetitive calculations. Several computer systems have been designed to help doctors to determine the optimum dose of drugs. We assessed the benefits of these systems to establish whether they should be used more widely.
We identified all comparative studies in which computers were used to help determine the most appropriate drug dose. The criteria for entry into the review were standard for reviews undertaken by the Cochrane Collaboration on Effective Professional Practice and include methodological and quality criteria for rigorous design of experimental and quasi-experimental studies.4
Methodological criteria were
Studies using any objective measure of patient outcome or provider behaviour, randomised or quasi-randomised by patient, doctor, practice, or provider of health care
Interrupted time series with a clearly defined intervention and at least three time points before and three after the intervention
Non-randomised studies controlled at a second site with data before and after the intervention and appropriate choice of control site.
We included all studies using a reliable, objective, predetermined measure of the process or outcome of health care. This includes studies comparing computer aided decisions either to unassisted decisions or to decisions made using aids such as nomograms, as well as studies in which the computer directly administered the drug to patients (such as with a computer controlled pump). We excluded studies in which the computer simply suggested giving or withholding a drug. The criteria were applied independently by two researchers and any disagreements were resolved by group discussion.
Relevant studies were located from the specialised register of studies of the Cochrane Collaboration on Effective Professional Practice.5 This register is updated by electronic searches and hand searches of relevant journals. We also located references through bibliographies of related topics and contact with experts and pharmaceutical companies. We made specific searches of Medline and Embase from 1966 to June 1996 to identify relevant references. The search terms were “computer assisted decision making” or(prescr* and comput*) and (“randomised controlled trial” or “random allocation” or “double blind method”). Search strategies were modifications of those designed to give a high yield of randomised controlled clinical trials.6 In addition, we hand searched issues of Therapeutic Drug Monitoring published from January 1993 to July 1996.
Outcome measures, determined in advance, were
Proportion of patients in which drug dose is changed because of computer advice
Proportion of patients with unwanted effects of treatment
Proportion of patients with blood concentrations of drug or a physiological measurement within the desired range
Differences in blood concentrations of drug or physiological measurements attributable to computer support
Time to achieve therapeutic control
Proportion of patients with improved outcome attributable to computer advice.
To these we subsequently added changes in the cost of treatment attributable to computer support.
Two researchers reviewed each study independently and, using a standard form, extracted data on methodology, outcomes, and quality criteria.4 We recorded the unit of allocation and analysis, concealment of allocation, blinding, statistical power, follow up of patients and professionals, baseline measurements, and protection of the control group from contamination by the intervention. We calculated the mean difference in outcome with computer support, as a percentage of the mean value without support for all outcomes fitting our criteria for inclusion. These relative percentage differences were used in the narrative part of the review.
Studies with comparable outcomes were divided into groups for meta-analysis according to outcome measure. Of the six predefined outcome measures, only four provided suitable data for meta-analysis: dose of drug; blood concentrations of drug; time to reach therapeutic concentration or effect; and difference between patients' drug concentration and target concentration. Four separate meta-analyses were performed.
We estimated effect sizes as standardised weighted mean differences for each outcome in each study where the relevant data were available. The effect size is a statistical measure of the impact of the intervention that is independent of the units used to measure an outcome. This measure quantifies the effect of an intervention in units of standard deviation and allows comparison of studies of the same intervention that measured different outcomes.
We used a random effects model to combine the effect sizes to give an overall effect for each subgroup of studies. This model was chosen because the outcomes we combined were for studies on different drugs and different diseases. The random effects model allows quantitative combination of outcomes but does not assume that all interventions have the same underlying effect. If the outcome was measured at different times in the same study, we selected the value nearest the midpoint of the intervention period. When there were related outcomes from the same study we used the mean of the effect sizes.7 In this way only a single effect size for each study was pooled. We performed calculations using the RevMan software provided by the Cochrane Collaboration.
Characteristics of included studies
We identified 23 relevant studies, of which 16 randomised controlled clinical trials8–23 and one non-randomised controlled clinical trial24 met the inclusion criteria. All used reliable outcome measures with adequate blinding of assessment. No interrupted time series or studies controlled at a second site were identified. Sixteen studies used patients as the unit of allocation; one allocated medical firms to intervention and control.9 Three studies did not use the same unit for allocation and analysis. 9 12 16 Only two studies reported a power calculation. 11 15 Six studies reported adequate concealment of allocation (for example, random numbers in opaque envelopes),8–13 12 followed up more than 80% of patients, 8 12 13 15–20 22 24 25 12 reported similar baseline measures between intervention and control groups, 8–14 16 1920 23 24 nine recorded that patient consent had been obtained, 8–10 15 18–20 22 23 and eight reported gaining approval from an ethics committee. 8 10 12 18–20 22 24 In one study the reviewers thought that there was little room for improvement because the performance of the health professional was adequate without the intervention.23
Most studies were randomised by patient, so the same health professional might have given treatment to study and control groups. If the same person treated both groups it is possible that the effect of computer advice might have carried over into the control group. Such studies would tend to underestimate the effect of computer support. Two studies of continuous infusion anaesthesia, although randomised by patient, had a sufficiently rigorous study design to ensure that this contamination was unlikely to occur. 19 24 In these studies a computer controlled pump delivered the drug directly to the patient, and the anaesthetist administered additional anaesthetic agents without knowing the amount of drug given by the computer. In the study randomised by medical firm all firms worked at the same hospital, so the computer advice might have influenced treatment of the control group. The only studies judged to be free of contamination were the two studies of continuous infusion anaesthesia.
Types of computer support systems used
Most of the computer systems used a mathematical model of the pharmacokinetics of the drug in question to predict the required dose (table). These models represent the compartments in the body in which the drug is distributed. Rate constants are used to describe the movements of the drug between different compartments. The models ranged from a simple, one compartment model for theophylline14 to a more complex, three compartment model for fentanyl.24 The starting values for the rate constants were estimated from population data but could then be adjusted as data accumulated from an individual patient. The systems allowed the operator to specify a target blood concentration of drug, which the computer then attempted to achieve. When the effect of the drug was more important than its blood concentration, pharmacodynamic parameters based on population data were added to the model.22
Effects of computer support on outcome
The table shows the effects of computer support on the process and outcome of care.
Drug doses used —Eleven studies examined change in the drug dose when computer support was used, 8 9 11 13 14 17–19 21 23 24 and seven found significant changes. 8 13 14 17 19 21 24Studies on theophylline showed increases in initial dose 14 21and in maintenance dose. 13 14 Studies on intravenous anaesthesia showed a reduction in total dose of fentanyl used24 and a reduction in initial dose and maintenance dose of midazolam.19
Drug concentrations within desired range —Of the seven studies that measured changes in drug concentrations in the body two found significant increases in the proportion of patients with drug concentrations in the therapeutic range with computer support. 9–11 13 14 17 21
Physiological control —Eight studies measured changes in control of a physiological parameter with computer support, 8 15 16 18 20 22–24 of which six showed significant benefit. 8 15 18 20 22 24 Computer support for anticoagulant control resulted in significant reductions in the time taken to achieve the desired prothrombin time22 and activated partial thromboplastin time.15 In postoperative control of blood pressure with sodium nitroprusside, a computer assisted pump was more effective at keeping blood pressure in the target range than a manually controlled infusion. Babies delivered to women treated with computer controlled oxytocin had a lower lactate concentration in the umbilical cord blood.8
Unwanted effects of drug treatment —Six studies measured the unwanted effects of drugs, 11 12 14 15 22 24 and four found significant reductions associated with computer support. 14 15 22 24 Fewer patients treated with theophylline reached toxic drug concentrations when computer advice was used.14 In studies on anticoagulation both the number of patients given too much anticoagulant22 and the total number of adverse events15 were reduced in the intervention groups. The number of hypotensive episodes during cardiac surgery was reduced when fentanyl and midazolam were given via a computer controlled pump.24
Cost of drug treatment —Only two studies reported economic data, and both looked at computer support for aminoglycoside dose. 9 11 In one study the mean direct cost of treatment with computer support was $7102 compared with $13 758 in controls (P<0.02), with a benefit to cost ratio of 75.11 The other calculated a cost avoidance (the money potentially saved by the intervention) of $1311 for each patient treated, with a benefit to cost ratio of 4.1.9 These cost savings resulted largely from reduced hospital stays, which was confirmed in one study,14 although another suggested an increased time spent in hospital.10 Another study showed that computer support lengthened the interval between outpatient visits.12
Outcome of medical care —Six studies directly measured outcomes of care, of which five showed benefits. Three showed significant benefits in clinical improvement scores for asthma,21 treating infection,9 and pain relief after surgery.20 Fewer caesarean sections were required when oxytocin was given by computer controlled pump to augment labour.9 With computer support for hospital treatment of acute asthma, fewer patients subsequently needed convalescent care,13 and there were fewer deaths.14 One study on anticoagulation showed an increase in the number of embolisms, but it may be that the computer system used in this study was set to achieve too low a prothrombin time.12
Overall effect —Eleven studies provided outcomes for quantitative synthesis, 10 11 13–15 17–19 21–23 and the figure shows the individual results and meta-analysis for each of the four outcome measures. Patients treated with computer support had higher blood concentrations of the drug (effect size 0.69, 95% confidence interval 0.36 to 1.02) and took less time to reach therapeutic concentrations −(0.44,−0.71 to 0.17). However, computer support had no significant effect on the total amount of drug used (−0.43, −1.00 to 0.15) nor on the difference between the level of a physiological parameter achieved and the target level (−1.22, −3.31 to 0.87). Although the clinical settings of the trials varied widely, only in the case of difference from target level was there evidence of statistical heterogeneity.
This review suggests that substantial benefit results from computer support for determining the dose of certain drugs in acute hospital settings. In the studies we identified, unaided doctors were often excessively cautious in estimating drug dose. This caution presumably resulted from an unwillingness to expose patients to adverse effects of drug treatment. Unaided doctors used lower initial doses and maintenance doses than when computer support was available. 14 21 Lower doses lead to lower blood concentrations and often to suboptimal therapeutic effects. Although doses with computer support tended to be higher than those used by unaided doctors, no studies reported an increase in unwanted effects due to overdose. This suggests that the computers helped doctors to tailor drug doses more accurately to individual patients. Higher initial doses with computer support gave more rapid therapeutic control, 20 22 bringing benefits for patients and reducing the time that they spent in hospital. 9 14 22 Unaided doctors tended to exercise caution in checking blood concentrations, which resulted in more blood tests15 and hospital visits.12
The most successful systems were those in which the computer administered drugs directly to patients under medical supervision. Manually controlled infusions resulted in higher doses of anaesthetic agents compared with computer controlled infusions. 19 24 This may result from the doctors' reluctance to expose patients to the risk of unnecessary pain. However, patients treated with computer support experienced less pain,20 suggesting that computer support could help doctors to adjust drug doses more accurately for individual patients.
Potentially, the most important factor that may undermine our analysis is publication bias—studies with positive results are more likely to be published than those with negative results.26 Our search was exhaustive, and it seems unlikely that large numbers of patients have been randomised to trials that we have not identified. Orwin's file drawer method suggests that there would need to be 24 unpublished studies showing no benefit from computer support to alter our results significantly.27
Another review in a similar area did not use statistical methods to estimate the overall effects from computer support,28 preferring to present a “vote count” of statistically significant studies. This method risks concluding falsely that an important effect is absent because it assumes that a trial with no significant effect is a negative study.29 The method also does not make full use of available data. Our overview represents an advance from this approach, using established and robust statistical methods to estimate an overall treatment effect. 29 30
Our findings need to be read with caution since we identified relatively few studies on a limited range of drugs and our quantitative analysis was based on results derived from only 671 patients. Computer assisted determination of drug dose is a potentially hazardous intervention: it is surprising that the risks and benefits have not been evaluated in large randomised controlled trials. The quality of the studies could be improved: most did not report a power calculation, and sample sizes were often small. A common bias in many of the studies was that the same clinicians treated patients allocated to intervention and control, so the effect of the intervention would tend to spill over into the control group. Contamination of the control group in this manner would tend to make it more difficult to show a benefit from computer support.
More large scale studies are needed to confirm our conclusions, and research is needed to examine the effects of computer support in general use and to develop and implement appropriate systems. Existing studies suggest that computer support for drug dosage will be cost effective, but economic evaluation should be an integral part of any future study. Economic benefits seen in one therapeutic area may not transfer to another clinical situation.
Implications for computer support for drug dose
The scope of computer support systems could be widened to include other, commonly prescribed drugs with a narrow therapeutic window, such as anticonvulsants and lithium. A computer might use the same basic pharmacokinetic model for several different drugs. It is possible that the model could be extended to predict the likelihood and severity of some interactions—for example, those caused by competition for protein binding sites.
A barrier to adopting computer support may be the lack of access to suitable computers and electronic medical records. Computerised records are used in some hospital departments (such as intensive care), but hospitals often rely on paper to record outpatient treatment. In contrast, most general practitioners routinely use electronic records for prescribing31 and have easy access to suitable hardware that could run programs to help them to determine the most appropriate dose of commonly used drugs. General practice computers often already store necessary data such as blood concentrations of drugs, body mass index, and indicators of hepatic and renal function. It may be that the benefits seen in secondary care could be realised on a large scale if programs giving support for drug dose were integrated with the software routinely used in general practice.
A parallel version of this review will appear in the Cochrane Library. We thank referees in the Cochrane Collaboration on Effective Professional Practice review group for helpful comments on the protocol for this review. Since this review was conducted, the Cochrane Collaboration on Effective Professional Practice has changed its name to the Cochrane Effective Practice and Organisation of Care Group (EPOC).
Contributors: All the authors contributed to writing the protocol and participated in dual, independent data extraction and in reviewing drafts of the paper. RW developed the hypothesis, initiated the study, coordinated the review process, and wrote the original draft of the paper. RW and NF conducted the meta-analyses with advice from Frederick Wolf. SD constructed the data tables and hand searched Therapeutic Drug Monitoring. Emma Harvey helped to refine the protocol and contributed extensively to revising the paper. Andy Oxman helped to refine the study question. Martin Vessey, Pat Yudkin, and Mike Murphy commented on earlier drafts of the paper. RW is guarantor for the article.
Funding NHS research and development programme evaluating methods to promote the implementation of research findings. RW is supported by a Royal College of General Practitioners/BUPA training fellowship. SD was supported by the New Zealand Health Research Council.
Conflict of interest None reported.