A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97BMJ 2001; 322 doi: http://dx.doi.org/10.1136/bmj.322.7281.279 (Published 03 February 2001) Cite this as: BMJ 2001;322:279
Objectives: To appraise findings from studies examining the impact of computers on primary care consultations.
Design: Systematic review of world literature from 1980 to 1997.
Data sources: 5475 references were identified from electronic databases (Medline, Science Citation Index, Social Sciences Citation Index, Index of Scientific and Technical Proceedings, Embase, OCLC FirstSearch Proceedings), bibliographies, books, identified articles, and by authors active in the field. 1892 eligible abstracts were independently rated, and 89 studies met the inclusion criteria.
Main outcome measures: Effect on doctors' performance and patient outcomes; attitudes towards computerisation.
Results: 61 studies examined effects of computers on practitioners' performance, 17 evaluated their impact on patient outcome, and 20 studied practitioners' or patients' attitudes. Computer use during consultations lengthened the consultation. Reminder systems for preventive tasks and disease management improved process rates, although some returned to pre-intervention levels when reminders were stopped. Use of computers for issuing prescriptions increased prescribing of generic drugs, and use of computers for test ordering led to cost savings and fewer unnecessary tests. There were no negative effects on those patient outcomes evaluated. Doctors and patients were generally positive about use of computers, but issues of concern included their impact on privacy, the doctor-patient relationship, cost, time, and training needs.
Conclusions: Primary care computing systems can improve practitioner performance, particularly for health promotion interventions. This may be at the expense of patient initiated activities, making many practitioners suspicious of the negative impact on relationships with patients. There remains a dearth of evidence evaluating effects on patient outcomes.
Information technology has rapidly become an important component of primary care.1 Its application to the administrative tasks required of a busy practice has already shown benefits such as in patient registration and production of practice profiles.2 Its potential contributions to patient management through access to reference information3 and the provision of decision support are more recent developments.4 Computers are moving into the consultation itself, and the government's pledge to create an “NHS information superhighway” by the end of 20025 means that they will play an increasingly important role.
The development of primary care computing requires rigorous evaluation of existing and emergent information and communication technologies.6 However, the focus and methodology of studies in this subject have been criticised: scoring systems have tended to concentrate on methodologies of randomised controlled trials, which are often inappropriate in such a fast changing environment where multiple approaches to evaluation may be required.7 We have extended our systematic review reported in 19958 and evaluated primary care computing systems from 1980 to 1997.
Systematic review of literature 1980-1997
We conducted a worldwide review of the literature, updating our earlier systematic review described elsewhere.8 Briefly, we searched for prospective studies that concerned doctors or nurses in a primary care setting and described any computing system designed for use by a doctor. We searched the databases of Medline, Science Citation Index, Social Sciences Citation Index, Index of Scientific and Technical Proceedings, Embase, and OCLC FirstSearch Proceedings. Non-English language journals were included in the search. We also reviewed books, bibliographies, and conference proceedings of related topics as well as citations in these books and articles and references provided by colleagues. The Cochrane Group for Effective Practice and Organisation of Care (EPOC) provided references of articles containing the term “computer,” and authors active in the field were asked about studies in progress and unpublished work.
Our selection criteria for studies were identical to those used in our previous review—that is, studies that examined the effects of computers on the consultation process, on general practitioners' task performance, and on patient outcomes. In this review, however, we were also interested in identifying potential barriers to effective implementation and use of computers, and we included studies that determined doctors' or patients' attitudes towards computerisation. We excluded studies on validation of data or administrative use. To determine the suitability of studies for inclusion, both authors independently reviewed the abstracts of articles and then, after exclusion of unsuitable studies, the full reprints. We discussed any differences in our evaluations and reached a consensus for each study.
Appraisal of studies
We assessed the methodological adequacy of randomised controlled trials, clinical trials, and controlled before and after studies using a scoring system based on that proposed by Johnson et al9 (see box on the BMJ's website for details). The constantly evolving nature of computer technology has resulted in a lack of controlled trials of their use, and strict adherence to Cochrane standards would exclude less rigorous but still useful studies. We developed a method of scoring non-experimental studies using a two round, Delphi survey to reach a consensus on the methodological criteria to include (see box on the BMJ's website). This well established decision making mechanism involves recruiting a panel of experts to agree about a particular set of hypothetical questions.10 The 10 experts chosen were selected because of their experience in primary care research, medical informatics, and critical appraisal. Each paper was given a score with the scoring system relevant to its study design.
Our search identified 5475 references. After exclusion of editorials, dentistry or veterinary medicine studies, and duplicates, 1892 remained for review of their abstracts. We obtained full reprints for 214 of these and included 89 studies in our review,w1-w89 an increase of 62 on our total in 1994 (27). The κ coefficient for inter-rater agreement beyond chance was 0.7.
Of these studies, 61 examined the effects of computers on practitioners' performance, 17 evaluated their impact on patient outcome, and 20 determined practitioners' or patients' attitudes (see tables on the BMJ's website for details). Nine studies examined more than one aspect. Fifty five studies were given an identical score by both reviewers, and we reached an agreed score for the others.
In our 1994 review we assessed methodological quality for 28 papers. One of these was excluded from this updated review on the basis that the computer was not specifically the intervention11; a second was recategorised as an assessment of attitudes towards computers.w79 This left 26 papers for comparison, and, for this update, we assessed a further 46 papers for methodological quality. The remaining 17 papers we identified dealt with attitudes towards computers only and were not assessed. The median score for quality in 1994 was 7/10 (interquartile range 5-8) and in 1997 was 6/10 (range 4-8). These scores are not significantly different (P=0.240), although the results in the updated review are more widespread.
Impact on practitioner performance
This was the most predominant topic and one in which most studies (41/61) used a controlled trial design. The main aspect of performance to be assessed was immunisation and prevention (30/61 in 1997 update, 14/27 in 1994). Other subjects covered were management of disease (11/61, 2/27), content of consultation (9/61, 6/27), and prescribing (7/61, 4/27).
Content of consultation—Studies focused mainly on consultation length and doctor-patient interaction. Six studies looked at consultation length. This increased by 48-130 seconds in five of the studies, although this increase declined after variable time periods.w38 w52-w54 w56 In one study doctors worked an average of 30 days before their consultation lengths returned to baseline levels.w38 The remaining study found no significant difference in consultation length for three of the four doctors studied.w25 Two studies found that doctors spent 11%-100% more time on computerised records than they had on conventional records.w25 w52 This was mainly because of increased administrative tasks and preventive issues prompted by computer use.w25 w53 w56 Computer use led to increases in doctor-centred speech and the number of medical topics raised, often at the expense of patient-centred activity.w54 w55 Practitioners were also less likely to continue interacting with patients when using computerised records than when using paper records,w25 w52 and this did not diminish with increased familiarity.w27 In an attempt to minimise this, patients in one study synchronised their speech with perceived pauses in practitioners' keyboard use.w27
Immunisation rates improved by 8-34%in the nine studies of this issue.w5 w13 w14 w18 w21 w29 w31 w33 w42 In eight studies reminder systems were used, and the greatest improvements in immunisation rates were seen with patient only remindersw5 w21 and reminders to both doctor and patient.w13 McDowell et al found that immunisation rates fell to levels similar to that of control practices when the reminders were stopped.w21
Performance of preventive tasks, such as blood pressure screening and cervical smears, improved by up to 47%.w2 w4 w6-w9 w11 w12 w16 w17 w22 w28 w34-w36 w40 w41 w44 w47 w48 w56 w58 The greatest increases occurred when practitioners were prompted as part of the consultation.w4 w8 w11 w36 w44 One study showed a 5% improvement in performance with a change from no prompting to a nurse initiated prompting system, and a further 10% increase with a computerised prompting system.w36 However, like immunisations, increased preventive activities fell to pre-intervention levels when reminders were no longer provided.w7 Studies evaluating reminders to patients also found increases of 2-30%.w6 w9 w40 However, two studies that assessed the effects of both doctor and patient reminders on mammography screening found no difference in uptake rates when only patients were sent a reminder.w16 w47
Disease management was also improved by use of computers. Four studies that evaluated standards of diabetes care found improvements of 5-69%.w10 w19 w37 w61 The greatest improvement occurred when physicians used an electronic protocol, although this increased the length of consultations by 10 minutes.w19 Studies evaluating hypertension management found improvements of 18-53% in examinations.w23 w46 Again, the largest increase occurred with an electronic protocol, and consultation length increased by 35%.w23 Computerised alerts and reminders to doctors for management of HIV infection produced faster response times.w24 However, computerised decision support for lipid management produced no real differences, and system use was less than expected.w43 The introduction of a computer algorithm for paediatrics increased recording and compliance with management plans, but doctors found it “too tedious to use during routine care” and the study was abandoned after five weeks.w26
Prescribing improved with computer support: prescribing of generic drugs increased,w1 and prescribing costs declined.w39 w59 w60 One study found significant reductions in doctors' and receptionists' time when computerised prescriptions were issued.w3 These elements combined show why prescribing remains the most commonly used feature of general practice computing. Computer use for ordering tests led to reductions of 6-75% in numbers of tests and cost savings of 8-14%.w20 w30 w32 w49
Impact on patient outcomes
Studies of patient outcome were less common (2/27 in 1994, 17/89 in 1997 update). Use of computers in management of hypertension significantly increased the number of patients with reduced diastolic pressure.w46 w62 w70 However, their usefulness for anticoagulation management was not as clear: one study found improvements of 32-66%,w63 while another found no difference.w71
The introduction of computers to the consultation did not lead to any increase in service use either in visits to primary care or in referrals to secondary care.w24 w30 w61 w64 One study, however, reported a slight shift in activity, with patients in computerised practices being managed more in the community.w61 An evaluation of computerised decision support for lipid management found a 55% reduction in the number of expected referrals.w43 This reduction in service use was evident where computers were used specifically for disease management or appointment scheduling, as numbers of visits to practitioners and rates of non-attendance could be reducedw62 w69 and recall periods extended.w63
Four studies on patient satisfaction detected no significant changes when computers were introduced.w61 w63 w67 w72
Practitioners' and patients' attitudes
Most practitioners willingly accepted computers as part of their working environment and were positive about their use.w73-w75 Many thought that computerised records were more accurate than conventional recordsw82 or that they improved patient care.w80 w84 Patients said that computers gave their doctors better access to recordsw76 w79 w84 and that consultations were unaffected.w76 w78 w84 w89
However, five themes emerged that could prove major barriers to successful implementation of computers: privacy, the doctor-patient relationship, cost, time, and training.
Loss of privacy and confidentiality was the commonest concern to patients. Many thought that computerised notes posed a greater threat to privacy and were more vulnerable to unauthorised access than conventional records,w76 w77 w79 w82 w84 w87 w88 and should therefore be restricted to non-sensitive information.w77 One study found that some patients were unwilling to be completely frank about their problems in front of doctors using computerised records and would consider changing to another doctor.w86
Both practitioners and patients were concerned about the possible negative impact of computers on the doctor-patient relationship.w52 w74-w76 w85 w87 w88 This concern was partly due to the logistics of incorporating a computer in the consultationw52 w85 and partly to the perception that computers would take over the doctor's role.w76
Costs of computerisation were considered prohibitive, both by practitionersw73 w75 w82 w81 and patients.w84 w87 Many doctors said that the time commitment involved in learning and using computers was too greatw75 w81 and was more than they had expected,w80 resulting in additional stress.w52 Finally, existing training in computer use was perceived as being poor,w82 and it was thought that this should be made a component of doctors' continuing medical education.w81
Most of the 89 studies in this review found positive effects of computerisation, showing, among other things, improvements in immunisations and preventive care and reductions in prescribing costs and unnecessary tests. Practitioners and patients were generally positive about computers, particularly in terms of access, accuracy, and the time saving properties of electronic patient records.
However, little has been done to alleviate fears of computers interfering in the consultation process and the doctor-patient relationship. We identified three new studies on consultation content for this latest review and again found that use of computers lengthened consultations. The proportion of time in a consultation that doctors spent not interacting with patients also increased, in one case by as much as 28%,w25 and this did not alter with improved proficiency in using computers. Another cause of anxiety for clinicians, and particularly for patients, was the issue of privacy and confidentiality of computerised records. Patients are not always made aware of the uses of information technology in primary care,w88 which may account for their ongoing concern over this issue.
One way to address these problems might be a programme of research on the best ways of integrating the computer into the consultation, starting with examples of current best practice and refining these in line with principles of effective communication.
Limitations of studies
Computerisation in the health service and in primary care in particular continues to increase, yet there remains a dearth of published evaluations into the impact of this technology. The greatest shortfall is in research on the impact of computers on patient outcomes: we identified only 17 studies on this subject. Although this is a considerable increase on our 1984-94 total of two, it is insufficient for what is almost certainly the most contentious issue about computerisation in any field of medicine—whether computers provide real benefits for patients.
Most of the outcome studies we examined used a rigorous methodology and rated highly for sample formation, adjusting for differences between the groups, and the objectivity of the outcomes being measured. However, they rarely considered the full implications of using the patient as the unit of analysis rather than randomising work units (such as practices). This may reflect the difficulties of researching a technology with which we are striving to keep up. Allocation by practice reduces the confounding effect of participating in research on those researched. However, computerisation in primary care is so widespread that finding practices which do not have the specific system feature to be evaluated as well as adequate controls is virtually impossible. Randomising practices to receive particular systems is also problematic. Not only is this expensive, but it often seems inconsequential; no sooner has the system been evaluated than it has been modified or updated and requires further evaluation.
The most fruitful areas of current research are preventive care, prescribing support, chronic disease monitoring, test ordering, and hospital referral. Few studies have dealt with nursing research in general practice, and little has been published on the impact of computer systems on other members of the primary care team.
It is over three decades since information technology was first introduced to primary care. In the 1960s its use centred on collating patient data; in the ‘70s the possibility of electronically linking primary and secondary care emerged; in the ‘80s computers were introduced to the consulting room; and in the ‘90s the internet provided the potential to obtain and review useful information during the consultation. After 30 years of analysing the “potential” benefits of computers, perhaps we should allow information technology in primary care to mature. In the 21st century we should accept that the computer is a useful tool. Rather than continually describing its capabilities, research must move forward to evaluate key outcomes for patients, practices, and the health service as a whole.12
What is already known on this topic
For most primary care consultations in Britain and elsewhere in the developed world, computers are available
When a computer is used during a consultation it can increase both the medical content and length of that consultation
What this study adds
Despite the rapidly changing nature of this technology and its capabilities, research has concentrated on preventive care and prescribing, with few studies evaluating patient outcomes
Research has centred on general practitioners, and little has been published on the impact of computers on other members of the primary care team
The main concerns of practitioners and patients about primary care computing are confidentiality, impact on the doctor-patient relationship, cost, time, and training
The results of this systematic review are also available in a MS Access database, which can be obtained on disk from E Mitchell.
We thank everyone who provided us with information for this review. We thank the expert panel who participated in our Delphi study, which we know was time consuming. We also thank Dr Sue Ross for her advice and Michere Beaumont for secretarial support.
Contributors: EM conducted the Delphi study, designed the review protocol and search strategy, conducted the literature retrieval, reviewed all abstracts identified, read all potentially relevant articles, scored all articles included in the review, and wrote the initial draft of the paper. FS reviewed all abstracts identified, read all potentially relevant articles, scored all articles included in the review, and contributed to and edited the paper.
Funding This study was funded by a grant from the Chief Scientist Office of the Scottish Executive Health Department (K/OPR/2/2D300).
Competing interests None declared.
References to the studies reviewed and tables giving details of methods and results appear on the BMJ's website. This article is part of the BMJ's trial of open peer review, and documentation relating to this also appears on the website