Martin Eccles and colleagues have performed a methodologically sound
study of a poorly developed intervention.1 We feel that this is a missed
opportunity in the development and evaluation of computer-based clinical
decision support systems (CDSSs) in UK primary care.
They quote a definition a computerised support system as being“a
system that compares patient characteristics with a knowledge base and
then guides a health provider by offering patient specific and situation
specific advice”,1 Unfortunately, the intervention developed and tested in
their study does not appear to meet these criteria: it did not depend on
patient-specific information but entry of a more general READ code; it did
not contain a reminder to initiate review of patient care or arrange
follow up; and it is unclear how far treatment recommendations depended on
the patient’s individual clinical review rather than issuing more generic
treatment recommendations.2;3
The PRODIGY system, the intervention around which this study was
based, is an electronic version of a paper guideline that is triggered by
entry of a pre-specified READ code. By making this the only way in which
to enter the computerised guideline the investigators ensured a low level
of use during the study. General Practitioners (GPs) are very unlikely to
continue to enter the same READ code at every consultation, as it would
have meant that each participating patient would have multiple duplicate
entries of the same READ code in their electronic record. By excluding any
sort of reminder function in their system,2 they have failed to account
for an important barrier that operates in chronic disease management,
namely registration, recall and regular review of patients. A “diagnostic
analysis” of factors that operate in the disease management of angina and
asthma should have uncovered such barriers prior to the start of this
study.4
Other details about the use of the computerised guideline require
clarification. Could the authors provide a definitive number of patients
randomised and followed up in each practice for each intervention? What is
the number (%) in whom the computer guideline went past the first screen?
What is the number (%) for whom a complete record entry was made? The
authors make no comment on the differential use of the electronic
guidelines between the two computer suppliers.
In conclusion, this study re-enforces the fact that passive diffusion
of guidelines- either in electronic or paper format, is an ineffective way
in which to implement best practice.4 Insufficient attention as to how a
computer interface operates has produced low levels of usage and made the
evaluation that was conducted less useful than it might have been.2;5
Future studies should be piloted and take into account the different
functions of CDSSs,5 rather than simply generation of suggestions to alter
prescribing practice.2 How do the developers of PRODIGY plan to modify
their system in the light of the findings from their RCT?
Yours sincerely,
Tom Fahey, Alan Montgomery, Liz Mitchell, Peter Gregor, Frank
Sullivan
References
1. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D et
al. Effect of computerised evidence based guidelines on management of
asthma and angina in adults in primary care: cluster randomised controlled
trial. BMJ 2002;325:941.
2. Randolph A, Haynes RB, Wyatt JC, Cook DJ, Guyatt GH. Users'
guides to the Medical Literature XVIII. How to use an article evaluating
the clinical impact of a computer-based clinical decision support system.
Journal of the American Medical Association 1999;282:67-74.
3. Hunt D, Haynes RB, Hanna S, Smith K. Effects of Computer-Based
Clinical Decision Support Systems on Physician Performance and Patient
Outcomes. Journal of the American Medical Association 1998;280:1339-46.
4. Anonymous. Getting evidence into practice. Effective Health Care
Bulletin 1999;5:1-16.
5. See Tai S, Nazareth I, Donegan C, Haines A. Evaluation of General
Practice Computer Templates: lessons from a Pilot Randomized Controlled
Trial. Methods of Information in Medicine 1999;38:177-81.
Competing interests:
None declared
Competing interests:
No competing interests
08 November 2002
Tom Fahey
Professor of Primary Care Medicine
Alan Montgomery, Liz Mitchell, Peter Gregor, Frank Sullivan
Tayside Centre of General Practice, University of Dundee, Kirsty Semple Way, Dundee DD2 4AD
Rapid Response:
A missed opportunity
Dear Sir,
Martin Eccles and colleagues have performed a methodologically sound
study of a poorly developed intervention.1 We feel that this is a missed
opportunity in the development and evaluation of computer-based clinical
decision support systems (CDSSs) in UK primary care.
They quote a definition a computerised support system as being“a
system that compares patient characteristics with a knowledge base and
then guides a health provider by offering patient specific and situation
specific advice”,1 Unfortunately, the intervention developed and tested in
their study does not appear to meet these criteria: it did not depend on
patient-specific information but entry of a more general READ code; it did
not contain a reminder to initiate review of patient care or arrange
follow up; and it is unclear how far treatment recommendations depended on
the patient’s individual clinical review rather than issuing more generic
treatment recommendations.2;3
The PRODIGY system, the intervention around which this study was
based, is an electronic version of a paper guideline that is triggered by
entry of a pre-specified READ code. By making this the only way in which
to enter the computerised guideline the investigators ensured a low level
of use during the study. General Practitioners (GPs) are very unlikely to
continue to enter the same READ code at every consultation, as it would
have meant that each participating patient would have multiple duplicate
entries of the same READ code in their electronic record. By excluding any
sort of reminder function in their system,2 they have failed to account
for an important barrier that operates in chronic disease management,
namely registration, recall and regular review of patients. A “diagnostic
analysis” of factors that operate in the disease management of angina and
asthma should have uncovered such barriers prior to the start of this
study.4
Other details about the use of the computerised guideline require
clarification. Could the authors provide a definitive number of patients
randomised and followed up in each practice for each intervention? What is
the number (%) in whom the computer guideline went past the first screen?
What is the number (%) for whom a complete record entry was made? The
authors make no comment on the differential use of the electronic
guidelines between the two computer suppliers.
In conclusion, this study re-enforces the fact that passive diffusion
of guidelines- either in electronic or paper format, is an ineffective way
in which to implement best practice.4 Insufficient attention as to how a
computer interface operates has produced low levels of usage and made the
evaluation that was conducted less useful than it might have been.2;5
Future studies should be piloted and take into account the different
functions of CDSSs,5 rather than simply generation of suggestions to alter
prescribing practice.2 How do the developers of PRODIGY plan to modify
their system in the light of the findings from their RCT?
Yours sincerely,
Tom Fahey, Alan Montgomery, Liz Mitchell, Peter Gregor, Frank
Sullivan
References
1. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D et
al. Effect of computerised evidence based guidelines on management of
asthma and angina in adults in primary care: cluster randomised controlled
trial. BMJ 2002;325:941.
2. Randolph A, Haynes RB, Wyatt JC, Cook DJ, Guyatt GH. Users'
guides to the Medical Literature XVIII. How to use an article evaluating
the clinical impact of a computer-based clinical decision support system.
Journal of the American Medical Association 1999;282:67-74.
3. Hunt D, Haynes RB, Hanna S, Smith K. Effects of Computer-Based
Clinical Decision Support Systems on Physician Performance and Patient
Outcomes. Journal of the American Medical Association 1998;280:1339-46.
4. Anonymous. Getting evidence into practice. Effective Health Care
Bulletin 1999;5:1-16.
5. See Tai S, Nazareth I, Donegan C, Haines A. Evaluation of General
Practice Computer Templates: lessons from a Pilot Randomized Controlled
Trial. Methods of Information in Medicine 1999;38:177-81.
Competing interests:
None declared
Competing interests: No competing interests