INFORMATION IN PRACTICE

Letters to the Editor

To the Editor:

We were very pleased to read the inaugural article in the Information in Practice series, "What Clinical Information do Doctors Need?"(1) We were especially pleased to see that you used our "Usefulness Equation"(2) as a proposed answer to clinicians' information needs. We would like to expand on our perspective by commenting on the low usefulness of some information sources commonly used by doctors.

Clinicians turn frequently to "expert-based" sources - colleagues, continuing medical education lectures, textbooks, and standard journal reviews - for new information. As the usefulness formula states:

Usefulness of medical information =
relevance x validity

work to access

These sources are potentially useful because the 'work' factor to access the information is low. The validity and relevance of the information that they provide may, however, be in doubt.

While most experts are very good at diagnosing disease and performing procedures due to their accumulated clinical experience, few are expert at evaluating the primary literature performing a 'validity' assessment - since critical appraisal is a recent addition to a minority of medical school curricula. As a result, expert-based therapeutic recommendations frequently rely only on clinical experience rather than a critical evaluation of the available evidence.

More importantly, the 'relevance' of the information that these sources provide also may be in question. Clinical evidence can be categorized as either "patient-oriented" or "disease-oriented."(3) POEMs are "Patient Oriented Evidence that Matters," evidence that considers outcomes that patients would care about (morbidity, mortality, quality of life) and that would "matter" because the interventions should change the way clinicians practice. DOEs are Disease Oriented Evidence, which is the large amount of intermediate or surrogate-endpoint information that makes up the knowledge base of almost all practicing clinicians. Reliance on DOE information is the main reason, therefore, that the majority of information found in expert-based systems is not relevant, and likely not very useful to either clinicians or their patients.

We applaud your efforts to help guide clinicians in the direction of more useful information.

David C. Slawson and Allen F. Shaughnessy

David C. Slawson, MD,
Associate Professor UVA-HSC,
Department of Family Medicine Box 414,
Charlottesville,
VA 22908

(voice) 804-924-1165
(fax) 804-982-4306
(email) dcs6e@virginia.edu

Allen F. Shaughnessy,
PharmD Harrisburg Family Practice Residency
P.O. Box 8500
Harrisburg,
PA 17105-8700

References

1. Smith R. What clinical information do doctors need? BMJ 1996;313:1062-8.

2. Shaughnessy A F, Slawson A F, Bennett J H. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract 1994;39:489-99.

3. Slawson D C, Shaughnessy A F, Bennett J H. Becoming a medical information master: Feeling good about not knowing everything. J Fam Pract 1994;38:505-513

Dear Editor (British Medical Journal):


I found the new British Medical Journal column, "Information in Practice" very complete, and very much on target (1). Knowing the questions that come up during a "typical" physician's day, and understanding how a physician might address those questions, is very important to several groups (e.g., medical educators, librarians, computer programmers).

The Center for Applied Medical Informatics at Michigan State University Kalamazoo Center for Medical Studies is devoted to understanding clinician needs towards the goal of designing usable and useful information systems.

My commentary on your column is based on our Center's experience and draws from the discipline of human factors engineering (i.e., ergonomics) (2). Human Factors Engineering research supports Richard Smith's assertion that simple surveys and self reports are out of context are almost useless to identify true physician information needs. Further, it is necessary to "probe" the physician-work environment with progressively refined prototypes to get an accurate handle on information needs and flow (3). Our center, and others (4, 5), have applied these techniques towards design of systems ranging from community health information networks to World Wide Web sites for rural practitioners in Michigan (USA).

For example, some medical software developers are using a group of techniques called Contextual Inquiry, which have been adopted by many information technology organizations in the United States and Europe (6). A World Wide Web site with resources and links regarding the application of human factors engineering to the medical domain is also available (7). Good luck in your very timely and practical column.

John Gosbee, MD, MS
Executive Director,
Center for Applied Medical Informatics
Michigan State University - Kalamazoo Center for Medical Studies
1000 Oakland Dr.
Kalamazoo,
MI 49008

616-337-4435 (ph)
616-337-4445 (fax)
gosbee@cps.msu.edu
http://omerad1.chm.msu.edu/Users/~omerad/Faculty/Gosbee.html

References

1 Smith R. Information in practice: What clinical information do doctors need. BMJ 1996;313:1062-8

2 Gosbee J W. Human factors engineering as part of quality improvement and reengineering. Proceedings of 1995 Annual Conference of the Healthcare Information and Management and Systems Society Conference. Chicago, IL: HIMSS. 1995;4:291-304.

3 Aucella A, Kirkham T, Barnhart S, Murphy L, LaConte K. Improving ultrasound systems by user-centered design. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting. Santa Barbara: CA: Human Factors and Ergonomics Society. 1994:XXX.

4 Coble J, Maffitt J, Orland M, Kahn M. (1995). "Contextual inquiry: Discovering physicians' true information needs. Proceedings of the Symposium on Computer Applications in Medical Care. 1995:XXX.

5 Fitter M, Brownbridge C, Garber B, Herzmark G. A human factors evaluation of the IBM Sheffield Primary Care System. Proceedings of IFIP INTERACT '84: Human-Computer Interaction. 1985:675-681.

6 Wixon D, Ramey J. Field Methods Casebook for Software Design. New York: John Wiley & Sons. 1996.

7 Medical Systems and Rehabilitation Technical Group of the Human Factors and Ergonomics Society. http://callan1.gtri.gatech.edu//msrtg/

Dear Editor -

I am a practicing primary care doctor and am surprised you fail to mention Mentor in your very comprehensive article on "Information in Practice"'. Mentor is a decision support system available free to EMIS sites. It will soon be available for purchase in Windows format for all.

Most textbooks are written by authors remote from the situation the questioner finds himself in, hence signal to noise ratio is low; whereas Mentor differs in that it is written in the most part by and for primary care doctors, based initially on the very successful Oxford handbooks.

These provide basic articles in standard format so information (e.g. Rx) can be accessed quickly. Links are provided (references or hypertext jumps) to related articles and more detailed reference texts. Initial information is limited to that which is required to make clinical decisions or give information to patients.

Access is quick in the consulting room, during the usual computer data entry that is increasingly part of the consultation. This involves a keypress from the Read Code picking list taking one directly to relevant articles, or a "look up" table.

General articles exist to help doctors through common diagnostic presentations and protocols (local or otherwise) can be incorporated into a user part of the text.

Where differences of opinion exist, all reasonable courses of action are stated.

Although many of the questions may be complex, such as the management of a patient with diabetes, renal failure and hypertension, they are seldom "new", and can in the most part be predicted by the authors. Mentor is only intended to take the doctor to the point of a specialist referral, and the information required for such a referral.

Feedback systems are in place so if questions are unanswered, this can be remedied ASAP by the authors. This is critical for the success of such a knowledge support system. It seems best to avoid the needless duplication of effort. The text on the main system is "patch updated" fortnightly. There will be links to other local CDROM texts and ultimately internet resources remotely via a dial-up link.

I have Mentor available on my portable which I take on visits.

Mentor has been used in our practice for over 2 years, particularly useful to GP registrars and assistants. I regularly use it during the consultations and patients seem reassured (rather than worried) when I look things up instantly when required.

Dr Huw Thomas DRCOG MRCGP
Mentor Peer Review Member

Irnham Lodge Surgery
Minehead
TA24 5RG

huw@irnham.demon.co.uk

Reference

1 Smith R Information in Practice. BMJ 1996;313:1062-8

Dear Editor -

Balint taught general practitioners over thirty years ago to respect the therapeutic effect they themselves had on their patients. Although applicable to all doctors, it was particularly relevant to general practitioners because of the volume of emotional malaise that they saw.

Your recent article(1) addresses comprehensively the secondary care viewpoint of the information needs of doctors. However, it pays scant attention to the practical needs of general practitioners (in the United Kingdom).

Much of the evidence you quote is based on studies from the United States, where the needs of primary care physicians are much more knowledge-based than the needs of general practitioners in the United Kingdom. Although in the UK we may have at least one question per consultation, it is as likely to be "Should I inform the mother of this wheezy child that the problem is asthma, or should I wait another couple of consultations to gain her confidence first?" as it is to be a question about therapeutic intervention. The model of a computer intruding on such a consultation demanding that I carry out a peak flow, measure height and issue an information leaflet would not help me, or the patient.

As a general practitioner, I do not need to spend precious time having to beat my computer back into its cage at each consultation. I need an information source that I can summon when I want it, can be retrieved within five seconds, and provides information relevant to me, not to tertiary care physicians. GP Software Suppliers are already trying to provide this. In my practice, the partners. nurses, and GP registrar use such a package. It is linked to Read clinical codes so that it will provide relevant information (only when I ask it!). it is updated automatically via modem, and all additions are peer-reviewed by general practitioners.

A final point relates to the formula described by Shaughnessy et al (2). General practitioners have to pay, to varying extents, for the provision of information. The drug company representative costs nothing. The evidence based, regularly updated textbook, much. Perhaps the formula should read

Usefulness of medical information =
relevance x validity

(cost to GP x work) to access

Yours faithfully

Dr Paul Scott

D'Evercy
Thorne Coffin
Yeovil
Somerset BA21 3PZ

Tel: 01935 33960
Fax: 01935 410188

References

1 Smith R. What clinical information do doctors need? BMJ 1996;313:1062-8

2 Shaughnessy A F, Slawson D C, Bennett J H. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract 1994;39:489-99