BMJ 1999;318:984-990 ( 10 April )

Information in practice

Computer support for determining drug dose: systematic review and meta-analysis

Robert Walton, research fellow, a Susan Dovey, visiting research fellow, a Emma Harvey, research fellow, b Nick Freemantle, senior research fellow.. c

a ICRF General Practice Research Group, University of Oxford, Department of Public Health and Primary Care, Institute of Health Sciences, Oxford OX3 7LF, b Department of Health Sciences and Clinical Evaluation, Alcuin College, York YO1 5DD, c Centre for Health Economics, University of York, York YO1 5DD

Correspondence to: Dr Walton robert.walton{at}public-health.oxford.ac.uk

    Abstract
Top
Abstract
Introduction
Methods
Results
Discussion
References

Objective: To review the effectiveness of computer support for determining optimum drug dose.
Design: Systematic review of comparative studies where computers gave advice to clinicians on the most appropriate drug dose. Search methods used were standard for the Cochrane Collaboration on Effective Professional Practice.
Subjects: Comparative studies conducted worldwide and published between 1966 and 1996.
Main outcome measures: For qualitative review, relative percentage differences were calculated to compare effects of computer support in different settings. For quantitative data, effect sizes were calculated and combined in meta-analyses.
Results: Eighteen studies met the inclusion criteria. The drugs studied were theophylline, warfarin, heparin, aminoglycosides, nitroprusside, lignocaine, oxytocin, fentanyl, and midazolam. The computer programs used individualised pharmacokinetic models to calculate the most appropriate dose. Meta-analysis of data from 671 patients showed higher blood concentrations of drug with computer support (effect size 0.69, 95% confidence interval 0.36 to 1.02) and reduced time to achieve therapeutic control (0.44, 0.17 to 0.71). The total dose of drug used was unchanged, and there were fewer unwanted effects of treatment. Five of six studies measuring outcomes of care showed benefit from computer assistance.
Conclusions: This review suggests that using computers to determine the correct dose of certain drugs in acute hospital settings is beneficial. Computers may give doctors the confidence to use higher doses when necessary, adjusting the drug dose more accurately to individual patients. Further research is necessary to evaluate the benefits in general use.


Key messages

  • This systematic review of studies examining computer support for determining optimum drug dose showed benefits from computer use

  • Computer support led to patients having increased blood concentrations of drug, reduced time to achieve therapeutic benefits, and fewer unwanted effects of treatment

  • Computer support helps doctors to tailor drug doses more closely to the needs of individual patients

  • All the studies took place in hospitals, and further research is needed to determine the risks and benefits of widespread use of computer support, particularly in general practice, where most prescribing takes place



    Introduction
Top
Abstract
Introduction
Methods
Results
Discussion
References

Maintaining therapeutic drug concentrations is a complex task requiring knowledge of medicine and pharmacokinetics, a good rapport with the patients, and some skill in calculating dose. Harm can be caused by miscalculating doses because many drugs have a narrow "window" in which therapeutic benefits can be obtained at a low risk of unwanted effects.

Monitoring drug treatment to optimise effects and minimise dangers can be time consuming and requires meticulous attention to detail. Doctors sometimes make errors of judgment because their capacity to process information is exceeded,1 and their computational skills may be inadequate to perform calculations about drug dose.2 For example, 82 of 150 hospital doctors were unable to calculate how many milligrams of lignocaine were in a 10 ml ampoule of 1% solution.3

Computers, however, are very good at gathering information and performing repetitive calculations. Several computer systems have been designed to help doctors to determine the optimum dose of drugs. We assessed the benefits of these systems to establish whether they should be used more widely.

    Methods
Top
Abstract
Introduction
Methods
Results
Discussion
References

Inclusion criteria
We identified all comparative studies in which computers were used to help determine the most appropriate drug dose. The criteria for entry into the review were standard for reviews undertaken by the Cochrane Collaboration on Effective Professional Practice and include methodological and quality criteria for rigorous design of experimental and quasi-experimental studies.4

Methodological criteria were

  • Studies using any objective measure of patient outcome or provider behaviour, randomised or quasi-randomised by patient, doctor, practice, or provider of health care
  • Interrupted time series with a clearly defined intervention and at least three time points before and three after the intervention
  • Non-randomised studies controlled at a second site with data before and after the intervention and appropriate choice of control site.

We included all studies using a reliable, objective, predetermined measure of the process or outcome of health care. This includes studies comparing computer aided decisions either to unassisted decisions or to decisions made using aids such as nomograms, as well as studies in which the computer directly administered the drug to patients (such as with a computer controlled pump). We excluded studies in which the computer simply suggested giving or withholding a drug. The criteria were applied independently by two researchers and any disagreements were resolved by group discussion.

Search strategy
Relevant studies were located from the specialised register of studies of the Cochrane Collaboration on Effective Professional Practice.5 This register is updated by electronic searches and hand searches of relevant journals. We also located references through bibliographies of related topics and contact with experts and pharmaceutical companies. We made specific searches of Medline and Embase from 1966 to June 1996 to identify relevant references. The search terms were "computer assisted decision making" or (prescr* and comput*) and ("randomised controlled trial" or "random allocation" or "double blind method"). Search strategies were modifications of those designed to give a high yield of randomised controlled clinical trials.6 In addition, we hand searched issues of Therapeutic Drug Monitoring published from January 1993 to July 1996.

Outcomes
Outcome measures, determined in advance, were

  • Proportion of patients in which drug dose is changed because of computer advice
  • Proportion of patients with unwanted effects of treatment
  • Proportion of patients with blood concentrations of drug or a physiological measurement within the desired range
  • Differences in blood concentrations of drug or physiological measurements attributable to computer support
  • Time to achieve therapeutic control
  • Proportion of patients with improved outcome attributable to computer advice.

To these we subsequently added changes in the cost of treatment attributable to computer support.

Data extraction
Two researchers reviewed each study independently and, using a standard form, extracted data on methodology, outcomes, and quality criteria.4 We recorded the unit of allocation and analysis, concealment of allocation, blinding, statistical power, follow up of patients and professionals, baseline measurements, and protection of the control group from contamination by the intervention. We calculated the mean difference in outcome with computer support, as a percentage of the mean value without support for all outcomes fitting our criteria for inclusion. These relative percentage differences were used in the narrative part of the review.

Quantitative analysis
Studies with comparable outcomes were divided into groups for meta-analysis according to outcome measure. Of the six predefined outcome measures, only four provided suitable data for meta-analysis: dose of drug; blood concentrations of drug; time to reach therapeutic concentration or effect; and difference between patients' drug concentration and target concentration. Four separate meta-analyses were performed.

We estimated effect sizes as standardised weighted mean differences for each outcome in each study where the relevant data were available. The effect size is a statistical measure of the impact of the intervention that is independent of the units used to measure an outcome. This measure quantifies the effect of an intervention in units of standard deviation and allows comparison of studies of the same intervention that measured different outcomes.

We used a random effects model to combine the effect sizes to give an overall effect for each subgroup of studies. This model was chosen because the outcomes we combined were for studies on different drugs and different diseases. The random effects model allows quantitative combination of outcomes but does not assume that all interventions have the same underlying effect. If the outcome was measured at different times in the same study, we selected the value nearest the midpoint of the intervention period. When there were related outcomes from the same study we used the mean of the effect sizes.7 In this way only a single effect size for each study was pooled. We performed calculations using the RevMan software provided by the Cochrane Collaboration.

    Results
Top
Abstract
Introduction
Methods
Results
Discussion
References

Characteristics of included studies
We identified 23 relevant studies, of which 16 randomised controlled clinical trials8-23 and one non-randomised controlled clinical trial24 met the inclusion criteria. All used reliable outcome measures with adequate blinding of assessment. No interrupted time series or studies controlled at a second site were identified. Sixteen studies used patients as the unit of allocation; one allocated medical firms to intervention and control.9 Three studies did not use the same unit for allocation and analysis. 9 12 16 Only two studies reported a power calculation. 11 15 Six studies reported adequate concealment of allocation (for example, random numbers in opaque envelopes),8-13 12 followed up more than 80% of patients, 8 12 13 15-20 22 24 25 12 reported similar baseline measures between intervention and control groups, 8-14 16 19 20 23 24 nine recorded that patient consent had been obtained, 8-10 15 18-20 22 23 and eight reported gaining approval from an ethics committee. 8 10 12 18-20 22 24 In one study the reviewers thought that there was little room for improvement because the performance of the health professional was adequate without the intervention.23

Most studies were randomised by patient, so the same health professional might have given treatment to study and control groups. If the same person treated both groups it is possible that the effect of computer advice might have carried over into the control group. Such studies would tend to underestimate the effect of computer support. Two studies of continuous infusion anaesthesia, although randomised by patient, had a sufficiently rigorous study design to ensure that this contamination was unlikely to occur. 19 24 In these studies a computer controlled pump delivered the drug directly to the patient, and the anaesthetist administered additional anaesthetic agents without knowing the amount of drug given by the computer. In the study randomised by medical firm all firms worked at the same hospital, so the computer advice might have influenced treatment of the control group. The only studies judged to be free of contamination were the two studies of continuous infusion anaesthesia.

Types of computer support systems used
Most of the computer systems used a mathematical model of the pharmacokinetics of the drug in question to predict the required dose (table). These models represent the compartments in the body in which the drug is distributed. Rate constants are used to describe the movements of the drug between different compartments. The models ranged from a simple, one compartment model for theophylline14 to a more complex, three compartment model for fentanyl.24 The starting values for the rate constants were estimated from population data but could then be adjusted as data accumulated from an individual patient. The systems allowed the operator to specify a target blood concentration of drug, which the computer then attempted to achieve. When the effect of the drug was more important than its blood concentration, pharmacodynamic parameters based on population data were added to the model.22


                              
View this table:
[in this window]
[in a new window]
 

Effects of computer support on the process and outcome of care (ordered by clinical problem)

Effects of computer support on outcome
The table shows the effects of computer support on the process and outcome of care.

Drug doses used ---Eleven studies examined change in the drug dose when computer support was used, 8 9 11 13 14 17-19 21 23 24 and seven found significant changes. 8 13 14 17 19 21 24 Studies on theophylline showed increases in initial dose 14 21 and in maintenance dose. 13 14 Studies on intravenous anaesthesia showed a reduction in total dose of fentanyl used24 and a reduction in initial dose and maintenance dose of midazolam.19

Drug concentrations within desired range ---Of the seven studies that measured changes in drug concentrations in the body two found significant increases in the proportion of patients with drug concentrations in the therapeutic range with computer support. 9-11 13 14 17 21

Physiological control ---Eight studies measured changes in control of a physiological parameter with computer support, 8 15 16 18 20 22-24 of which six showed significant benefit. 8 15 18 20 22 24 Computer support for anticoagulant control resulted in significant reductions in the time taken to achieve the desired prothrombin time22 and activated partial thromboplastin time.15 In postoperative control of blood pressure with sodium nitroprusside, a computer assisted pump was more effective at keeping blood pressure in the target range than a manually controlled infusion. Babies delivered to women treated with computer controlled oxytocin had a lower lactate concentration in the umbilical cord blood.8

Unwanted effects of drug treatment ---Six studies measured the unwanted effects of drugs, 11 12 14 15 22 24 and four found significant reductions associated with computer support. 14 15 22 24 Fewer patients treated with theophylline reached toxic drug concentrations when computer advice was used.14 In studies on anticoagulation both the number of patients given too much anticoagulant22 and the total number of adverse events15 were reduced in the intervention groups. The number of hypotensive episodes during cardiac surgery was reduced when fentanyl and midazolam were given via a computer controlled pump.24

Cost of drug treatment ---Only two studies reported economic data, and both looked at computer support for aminoglycoside dose. 9 11 In one study the mean direct cost of treatment with computer support was $7102 compared with $13 758 in controls (P<0.02), with a benefit to cost ratio of 75.11 The other calculated a cost avoidance (the money potentially saved by the intervention) of $1311 for each patient treated, with a benefit to cost ratio of 4.1.9 These cost savings resulted largely from reduced hospital stays, which was confirmed in one study,14 although another suggested an increased time spent in hospital.10 Another study showed that computer support lengthened the interval between outpatient visits.12

Outcome of medical care ---Six studies directly measured outcomes of care, of which five showed benefits. Three showed significant benefits in clinical improvement scores for asthma,21 treating infection,9 and pain relief after surgery.20 Fewer caesarean sections were required when oxytocin was given by computer controlled pump to augment labour.9 With computer support for hospital treatment of acute asthma, fewer patients subsequently needed convalescent care,13 and there were fewer deaths.14 One study on anticoagulation showed an increase in the number of embolisms, but it may be that the computer system used in this study was set to achieve too low a prothrombin time.12

Overall effect ---Eleven studies provided outcomes for quantitative synthesis, 10 11 13-15 17-19 21-23 and the figure shows the individual results and meta-analysis for each of the four outcome measures. Patients treated with computer support had higher blood concentrations of the drug (effect size 0.69, 95% confidence interval 0.36 to 1.02) and took less time to reach therapeutic concentrations (-0.44, -0.71 to 0.17). However, computer support had no significant effect on the total amount of drug used (-0.43, -1.00 to 0.15) nor on the difference between the level of a physiological parameter achieved and the target level (-1.22, -3.31 to 0.87). Although the clinical settings of the trials varied widely, only in the case of difference from target level was there evidence of statistical heterogeneity.



View larger version (33K):
[in this window]
[in a new window]
  		Quantitative effects of computer support for determining drug dose on outcomes of care (standardised weighted mean differences (95% confidence intervals) for computer support versus control)


    Discussion
Top
Abstract
Introduction
Methods
Results
Discussion
References

This review suggests that substantial benefit results from computer support for determining the dose of certain drugs in acute hospital settings. In the studies we identified, unaided doctors were often excessively cautious in estimating drug dose. This caution presumably resulted from an unwillingness to expose patients to adverse effects of drug treatment. Unaided doctors used lower initial doses and maintenance doses than when computer support was available. 14 21 Lower doses lead to lower blood concentrations and often to suboptimal therapeutic effects. Although doses with computer support tended to be higher than those used by unaided doctors, no studies reported an increase in unwanted effects due to overdose. This suggests that the computers helped doctors to tailor drug doses more accurately to individual patients. Higher initial doses with computer support gave more rapid therapeutic control, 20 22 bringing benefits for patients and reducing the time that they spent in hospital. 9 14 22 Unaided doctors tended to exercise caution in checking blood concentrations, which resulted in more blood tests15 and hospital visits.12

The most successful systems were those in which the computer administered drugs directly to patients under medical supervision. Manually controlled infusions resulted in higher doses of anaesthetic agents compared with computer controlled infusions. 19 24 This may result from the doctors' reluctance to expose patients to the risk of unnecessary pain. However, patients treated with computer support experienced less pain,20 suggesting that computer support could help doctors to adjust drug doses more accurately for individual patients.

Methodological issues
Potentially, the most important factor that may undermine our analysis is publication bias---studies with positive results are more likely to be published than those with negative results.26 Our search was exhaustive, and it seems unlikely that large numbers of patients have been randomised to trials that we have not identified. Orwin's file drawer method suggests that there would need to be 24 unpublished studies showing no benefit from computer support to alter our results significantly.27

Another review in a similar area did not use statistical methods to estimate the overall effects from computer support,28 preferring to present a "vote count" of statistically significant studies. This method risks concluding falsely that an important effect is absent because it assumes that a trial with no significant effect is a negative study.29 The method also does not make full use of available data. Our overview represents an advance from this approach, using established and robust statistical methods to estimate an overall treatment effect. 29 30

Our findings need to be read with caution since we identified relatively few studies on a limited range of drugs and our quantitative analysis was based on results derived from only 671 patients. Computer assisted determination of drug dose is a potentially hazardous intervention: it is surprising that the risks and benefits have not been evaluated in large randomised controlled trials. The quality of the studies could be improved: most did not report a power calculation, and sample sizes were often small. A common bias in many of the studies was that the same clinicians treated patients allocated to intervention and control, so the effect of the intervention would tend to spill over into the control group. Contamination of the control group in this manner would tend to make it more difficult to show a benefit from computer support.

More large scale studies are needed to confirm our conclusions, and research is needed to examine the effects of computer support in general use and to develop and implement appropriate systems. Existing studies suggest that computer support for drug dosage will be cost effective, but economic evaluation should be an integral part of any future study. Economic benefits seen in one therapeutic area may not transfer to another clinical situation.

Implications for computer support for drug dose
The scope of computer support systems could be widened to include other, commonly prescribed drugs with a narrow therapeutic window, such as anticonvulsants and lithium. A computer might use the same basic pharmacokinetic model for several different drugs. It is possible that the model could be extended to predict the likelihood and severity of some interactions---for example, those caused by competition for protein binding sites.

A barrier to adopting computer support may be the lack of access to suitable computers and electronic medical records. Computerised records are used in some hospital departments (such as intensive care), but hospitals often rely on paper to record outpatient treatment. In contrast, most general practitioners routinely use electronic records for prescribing31 and have easy access to suitable hardware that could run programs to help them to determine the most appropriate dose of commonly used drugs. General practice computers often already store necessary data such as blood concentrations of drugs, body mass index, and indicators of hepatic and renal function. It may be that the benefits seen in secondary care could be realised on a large scale if programs giving support for drug dose were integrated with the software routinely used in general practice.

    Acknowledgments

A parallel version of this review will appear in the Cochrane Library. We thank referees in the Cochrane Collaboration on Effective Professional Practice review group for helpful comments on the protocol for this review. Since this review was conducted, the Cochrane Collaboration on Effective Professional Practice has changed its name to the Cochrane Effective Practice and Organisation of Care Group (EPOC).

Contributors: All the authors contributed to writing the protocol and participated in dual, independent data extraction and in reviewing drafts of the paper. RW developed the hypothesis, initiated the study, coordinated the review process, and wrote the original draft of the paper. RW and NF conducted the meta-analyses with advice from Frederick Wolf. SD constructed the data tables and hand searched Therapeutic Drug Monitoring. Emma Harvey helped to refine the protocol and contributed extensively to revising the paper. Andy Oxman helped to refine the study question. Martin Vessey, Pat Yudkin, and Mike Murphy commented on earlier drafts of the paper. RW is guarantor for the article.

    Footnotes

Funding: NHS research and development programme evaluating methods to promote the implementation of research findings. RW is supported by a Royal College of General Practitioners/BUPA training fellowship. SD was supported by the New Zealand Health Research Council.

Competing interest: None reported.

    References
Top
Abstract
Introduction
Methods
Results
Discussion
References

  1. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectibility of man. N Engl J Med 1976; 295: 1351-1355[Abstract].
  2. Baldwin L. Calculating drug doses. BMJ 1995; 310: 1154[Free Full Text].
  3. Rolfe S, Harper NJN. Ability of hospital doctors to calculate drug doses. BMJ 1995; 310: 1173-1174[Free Full Text].
  4. Cochrane Collaboration on Effective Professional Practice. The Cochrane Collaboration on Effective Professional Practice data collection checklist. York: Department of Health Sciences and Clinical Evaluation, University of York , 1996.
  5. Bero L, Grilli R, Grimshaw J, Oxman A. The Cochrane Collaboration on Effective Professional Practice (CCEPP) module of the Cochrane database of systematic reviews [updated 03 June 1997]. In: Cochrane Collaboration ed. The Cochrane Library. Issue 3. Oxford: Update Software , 1997.
  6. Dickerson K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. In: Chalmers I, Altman D, eds. Systematic reviews. London: BMJ Publishing Group, 1995:17-36.
  7. Rosenthal R, Rubin D. Meta analytic procedures for combining studies with multiple effect sizes. Psychol Bull 1986; 99: 400-406.
  8. Willcourt RJ, Pager D, Wendel J, Hale RW. Induction of labor with pulsatile oxytocin by a computer-controlled pump. Am J Obstet Gynecol 1994; 170: 603-608[Medline].
  9. Burton ME, Ash CL, Hill Jr DP, Handy T, Shepherd MD, Vasko MR. A controlled trial of the cost benefit of computerized bayesian aminoglycoside administration. Clin Pharmacol Ther 1991; 49: 685-694[Medline].
  10. Casner PR, Reilly R, Ho H. A randomized controlled trial of computerized pharmacokinetic theophylline dosing versus empiric physician dosing. Clin Pharmacol Ther 1993; 53: 684-690[Medline].
  11. Destache CJ, Meyer SK, Rowley KM. Does accepting pharmacokinetic recommendations impact hospitalization? A cost-benefit analysis. Ther Drug Monit 1990; 12: 427-433[Medline].
  12. Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, et al. A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin. National Consortium of Anticoagulation Clinics. J Gen Intern Med 1994; 9: 131-139[Medline].
  13. Gonzalez ER, Vanderheyden BA, Ornato JP, Comstock TG. Computer-assisted optimization of aminophylline therapy in the emergency department. Am J Emerg Med 1989; 7: 395-401[Medline].
  14. Hurley SF, Dziukas LJ, McNeil JJ, Brignell MJ. A randomized controlled clinical trial of pharmacokinetic theophylline dosing. Am Rev Respir Dis 1986; 134: 1219-1224[Medline].
  15. Mungall DR, Anbe D, Forrester PL, Luoma T, Genovese R, Mahan J, et al. A prospective randomized comparison of the accuracy of computer-assisted versus GUSTO nomogram-directed heparin therapy. Clin Pharmacol Ther 1994; 55: 591-596[Medline].
  16. Poller L, Wright D, Rowlands M. Prospective comparative study of computer programs used for management of warfarin. J Clin Pathol 1993; 46: 299-303[Abstract/Free Full Text].
  17. Rodman JH, Jelliffe RW, Kolb E, Tuey DB, de Guzman MF, Wagers PW, et al. Clinical studies with computer-assisted initial lignocaine therapy. Arch Intern Med 1984; 144: 703-709[Abstract].
  18. Ruiz R, Borches D, Gonzalez A, Corral J. A new sodium-nitroprusside-infusion controller for the regulation of arterial blood pressure. Biomed Instrum Technol 1993; 27: 244-251[Medline].
  19. Theil DR, Stanley TED, White WD, Goodman DK, Glass PS, Bai SA, et al. Midazolam and fentanyl continuous infusion anesthesia for cardiac surgery: a comparison of computer-assisted versus manual infusion systems. J Cardiothorac Vasc Anesth 1993; 7: 300-306[Medline].
  20. Van den Nieuwenhuyzen MC, Engbers FH, Burm AG, Vletter AA, van Kleef JW, Bovill JG. Computer-controlled infusion of alfentanil versus patient-controlled administration of morphine for postoperative analgesia: a double-blind randomized trial. Anesth Analg 1995; 81: 671-679[Abstract].
  21. Verner D, Seligmann H, Platt S, Dany S, Almog S, Zulty L, et al. Computer assisted design of a theophylline dosing regimen in acute bronchospasm: serum concentrations and clinical outcome. Eur J Clin Pharmacol 1992; 43: 29-33[Medline].
  22. White RH, Hong R, Venook AP, Daschbach MM, Murray W, Mungall DR, et al. Initiation of warfarin therapy: comparison of physician dosing with computer-assisted dosing. J Gen Intern Med 1987; 2: 141-148[Medline].
  23. White RH, Mungall D. Outpatient management of warfarin therapy: comparison of computer-predicted dosage adjustment to skilled professional care. Ther Drug Monit 1991; 13: 46-50[Medline].
  24. Alvis JM, Reves JG, Govier AV, Menkhaus PG, Henling CE, Spain JA, et al. Computer-assisted continuous infusions of fentanyl during cardiac anesthesia: comparison with a manual method. Anesthesiology 1985; 63: 41-49[Medline].
  25. White KS, Lindsay A, Pryor TA, Brown WF, Walsh K. Application of a computerized medical decision-making process to the problem of digoxin intoxication. J Am Coll Cardiol 1984; 4: 571-576[Abstract].
  26. Freemantle N, Haines A, Mason J, Eccles M. CONSORT---an important step towards evidence based health care. Ann Intern Med 1997; 126: 81-83[Free Full Text].
  27. Wolf F. Meta-analysis: quantitative methods for research synthesis. Newbury Park: Sage , 1986.
  28. Johnston ME, Langton KB, Haynes B, Mathieux A. Effects of a computer-based clinical decision support system on clinician performance and patient outcome. Ann Intern Med 1994; 120: 135-142[Abstract/Free Full Text].
  29. Hedges L, Olkin I. Statistical methods for meta-analysis. London: Academic Press , 1985.
  30. Der Simonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177-188[Medline].
  31. Social Surveys (Gallup Poll). In: Computerisation in GP practices 1993 survey. London: Department of Health (Management Executive) , 1993.

(Accepted 9 November 1998)

© BMJ 1999

Related Articles

Research questions for systematic reviews must be unambiguous from protocol stage
Gunther Eysenbach
BMJ 1999 319: 1265. [Extract] [Full Text]

Computer support for determining drug doses improves outcomes
BMJ 1999 318: 0. [Full Text]

This article has been cited by other articles:

  • van der Sijs, H., Aarts, J., Vulto, A., Berg, M. (2006). Overriding of Drug Safety Alerts in Computerized Physician Order Entry. J. Am. Med. Inform. Assoc. 13: 138-147 [Abstract] [Full text]  
  • Garland, A. (2005). Improving the ICU: Part 2. Chest 127: 2165-2179 [Abstract] [Full text]  
  • Kawamoto, K., Houlihan, C. A, Balas, E A., Lobach, D. F (2005). Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 330: 765- [Abstract] [Full text]  
  • Nash, I. S., Rojas, M., Hebert, P., Marrone, S. R., Colgan, C., Fisher, L. A., Caliendo, G., Chassin, M. R. (2005). Reducing Excessive Medication Administration in Hospitalized Adults With Renal Dysfunction. American Journal of Medical Quality 20: 64-69 [Abstract]  
  • Trivedi, M. H., Kern, J. K., Grannemann, B. D., Altshuler, K. Z., Sunderajan, P. (2004). A Computerized Clinical Decision Support System as a Means of Implementing Depression Guidelines. Psychiatr. Serv. 55: 879-885 [Abstract] [Full text]  
  • Tamblyn, R., Huang, A., Perreault, R., Jacques, A., Roy, D., Hanley, J., McLeod, P., Laprise, R. (2003). The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ 169: 549-556 [Abstract] [Full text]  
  • McQueen, M. J. (2001). Overview of Evidence-based Medicine: Challenges for Evidence-based Laboratory Medicine. Clin. Chem. 47: 1536-1546 [Abstract] [Full text]  
  • Eysenbach, G. (1999). Research questions for systematic reviews must be unambiguous from protocol stage. BMJ 319: 1265a-1265 [Full text]  

Rapid Responses:

Read all Rapid Responses

Moving targets and a insufficient database search in a systematic review?
Gunther Eysenbach
bmj.com, 22 Apr 1999 [Full text]
Doctors' innumeracy needs tackling
C Bell
bmj.com, 6 May 1999 [Full text]

Student BMJ

Risk of surgery for inflammatory bowel disease: record linkage studies

What can you learn from this BMJ paper? Read Leanne Tite's Paper+

www.student.bmj.com

Listen to the latest BMJ Interview