Education and debate

Recent Advances: Medical informatics

^a Hewlett-Packard Research Laboratories, Stoke Gifford, Bristol BS12 6QZ (email: ewc@hplb.hpl.hp.com)

Medical informatics is as much about computers as cardiology is about stethoscopes. For those who have studied the application of information technologies in medicine, the past decade has delivered one unassailable lesson. Any attempt to use information technology will fail dramatically when the motivation is the application of technology for its own sake rather than the solution of clinical problems.^{1 2 3}

The role of the information sciences in medicine continues to grow, and the past few years have seen informatics begin to move into the mainstream of clinical practice. The scope of this field is, however, enormous. Informatics finds application in the design of decision support systems for practitioners,⁴ in the development of computer tools for research,⁵ and in the study of the very essence of medicine--its corpus of knowledge.⁶ The study of informatics in the next century will probably be as fundamental to the practice of medicine as the study of anatomy has been this century.

I will consider recent advances in medical informatics with two seemingly contradictory themes in mind--apparently unbridled technological promise against less than satisfying practical achievement--and against three criteria--possibility, practicability, and desirability. Possibility reflects the science of information--what in theory can be achieved? Practicability addresses the potential for successfully engineering a system--what can be built given the constraints of the real world? Desirability looks at the fundamental motivation for using a given technology.

These criteria are suggested because a framework is necessary to judge the claims made for these new technologies by those who seek to profit from them. Just as there is a longstanding symbiosis between the pharmaceutical industry and medicine, there is a newer and consquently less examined relation between medicine and the computing and telecommunication industries. Clinicians should try to judge the claims of these newcomers in the same cautious way that they would examine claims about a new drug.⁷ Perhaps more so, given that clinicians are far more knowledgeable about pharmacology that they are about informatics and telecommunications.

In this article I will first review recent activities in telemedicine. Since this is a new subject, research themes are only just becoming apparent. Then I will discuss protocol based decision support systems, which may be the first substantive clinical information systems to appear in routine clinical practice. Finally, I will examine the current state of clinical coding. The terminology and coding enterprise is a concerted attempt to describe uniformly the structure, content, and nature of medical knowledge.

Telemedicine

Definitions of telemedicine abound. The essence of telemedicine is the exchange of information at a distance, whether that information is voice, an image, elements of a medical record, or commands to a surgical robot. It seems reasonable to think of telemedicine as the communication of information to facilitate clinical care. And it is not a new enterprise--Einthoven experimented with telephone transmissions using his new invention, the electrocardiograph, at the beginning of the century.⁸

Recent advances in medical informatics
---------------------------------------------------
* The application of information and communication
technologies in health care should be
problem rather than technology driven
* The use of existing communication technologies
such as mobile telephones, voice mail,
and email may significantly improve delivery of
health care
* Research is needed to understand human
communication processes and needs in health
care
* Clinical information systems have an important
role in the delivery of evidence based
clinical practice
* The appropriate use of computerised protocols
can significantly improve clinical outcome
* Universal and complete clinical coding
schemes are unrealisable, and the continued
modification of centralised thesauruses may be
technically and financially unsustainable
* Multiple task specific terminologies developed
in conjunction with clinical protocols may
offer the most effective and maintainable long
term strategy

At its inception telemedicine was essentially about providing communication links between medical experts in remote locations. The health care system, however, is clearly inefficient because of its poor communication infrastructure and telemedicine is now seen as a critical way of reducing that cost. One estimate suggests that the health system in the United States could save $30 billion a year with improved telecommunications.⁹ Consequently, telemedicine has now become an important subject for research and development. As might be expected, the renewed interest in telemedicine also has much to do with the excitement of new technologies. Currently, the press is flooded with articles about the information superhighway, the internet (box 1 and appendix), and the rapid growth in the use of mobile telephones. Telemedicine is often presented in the guise of sophisticated new communications technology for specialist activities such as teleradiology and telepathology. These are championed by telecommunication companies because they have the potential to become highly profitable businesses for them.¹⁰ Perhaps influenced by these forces, much of the research in telemedicine is driven by the possibilities of technology rather than the needs of clinicians and patients.

Box 1--The internet
The Internet is an open and unregulated community of people who communicate freely
across an international electronic computer network. The number of medical sites
joining the internet increases monthly, as does the number of information resources
available on it.
  There is now good evidence that such services are valuable and in constant use. The
OncoLink information resource, for example, provides oncologists with up to date trial
and treatment information, as well as acting as an educational resource for patients and
their families. OncoLink was reportedly accessed 36000 times in March 1994.11 The
figure for April 1995 was 412365 accesses.
  The World Wide Web (or web) is perhaps the most important innovation on the
internet in the past few years. It is a software layer that provides users with a simple way
of accessing information. The web allows users to create and exchange text, image, and
video documents. The quality of these documents is now so high that the web is used
by some medical educational institutions. The University of Utah, for example, has an
extensive library of anatomical pathology images called WebPath for its students.
The National Library of Medicine's visible human project aims to create a complete,
anatomically detailed, three dimensional representation of the male and female human
body and to make this available on the internet. The project is collecting transverse
computed tomograms, magnetic resonance images, and cryoscopic sections taken at
1 mm intervals in male and female cadavers.
  Medical research is also taking advantage of the web as journals begin to appear on
the web in preference to or in advance of print. For example, the Journal of Medical
Imaging will publish papers on the net. Medical imaging is developing so quickly that
printed media are now seen only as the archival form of knowledge. The form best
suited to rapid dissemination is electronic, with the additional advantage that one can
create papers which contain text, graphics, sound, and moving images. Similarly, the
move to evidence based medicine will be able to use the web for rapid distribution of
important clinical management data. The Cochrane Collaboration, which seeks to
collect, review, and disseminate high quality overviews of the effects of health care, has
already set up a publicly accessible resource on the web.12

Yet the communications infrastructure used by health care will not need to be special. The telecommunications market is competitive and the evolving options are numerous. Health care providers will be able to use the services of cable television, mobile cellular carriers, and telecommunication companies. Furthermore, communications technology does not need to be sophisticated to deliver benefit. Appropriate use of today's telephone can make significant improvements to the delivery of care. For example, follow up of patients is often possible on the telephone.¹³ Rapid communication of hospital discharge information using existing electronic data transfer mechanisms is beneficial for general practitioners.¹⁴ The combined use of mobile telephones and paging systems can reduce the 5-10 minutes out of every hour many clinicians spend answering pagers.¹⁵

Perhaps more interestingly, inexpensive voice messaging systems can deliver simple but powerful services over existing telephone networks. Voice mail for example, has significant potential for improving the process of care (box 2).¹⁶ Leirer et al used a voice mail system to phone reminders about drug treatment automatically to elderly people at home, and they showed that it reduced both tardines and complete forgetting.¹⁷ As more patients get access to electronic mail, this will offer further avenues for innovative health services (box 3). Already in some populations, access to electronic mail is high. Fridsma et al in California found that 46% of their patients at clinic already used email, 89% of which was through their place of work.¹⁸

Box 2--Electronic mail
Electronic mail, or email, is typically used to send
short textual messages between computer users. It is
one of a number of electronic data exchange services
available to those with access to a computer network.
Email is usually available on networks that service
local campuses such as universities or hospitals. Email
can also be exchanged internationally if a campus
network is attached to a global network such as the
internet.

Box 3--Voice mail
Voice mail systems allow a telephone user to record,
store, or send spoken messages. Such messages can be
sent when it is more convenient to use a recording than
to speak directly. For example, the person called may
be unavailable, in a different time zone, or the message
may not warrant an interruption. Since messages are
stored on a computer, they can be retrieved,
manipulated, or forwarded. Thus a single message--for
example, a dictated radiology report--could be
automatically broadcast to several people. Typically,
each user will have a mailbox where new messages are
received. The mailbox will also store their prerecorded
greeting used when their telephone is unattended.
Voice mail allows outside callers to leave messages,
much like a telephone answering machine.

All these points suggest that the potential for the clinical application of communication technologies is indeed great, but equally that there is much still to learn. In particular, the relation between telemedicine and informatics needs to be explored in greater detail. Informatics focuses on the use of information, and telemedicine on its communication. Although seemingly disparate endeavours, they are intimately linked since the goals of communicating information and deciding on its content cannot be separated.¹⁹ Furthermore, there is little clinical value in information systems built simply to gather data for administrators without remembering that the essence of delivering health care is the communication of information between members of the clinical team. Together, the technologies of information and communication can enhance access to information, whether it is stored electronically or is in the possession of a colleague.

RESEARCH ISSUES

Several key research questions are apparent. Firstly, clinical practice already revolves around communication, often by telephone, and important information exchanged in this way is often lost because it is not documented.²⁰ Capturing the informal information currently lost in health care's communication channels may soon become an important issue for those developing the formal electronic patient record. Deciding what information is important and how that information is made available will require the resolution of issues of confidentiality and security, as well as the technology of storage and retrieval of voice recordings.

Secondly, people's understanding of the effects of technology on communication is still in its infancy. Researchers in human-computer interaction believe that before these technologies can be successfully introduced, the way in which people communicate needs to be understood.¹⁹ In one recent study the presence of a computer during doctor-patient consultations had detectable negative effects on the way doctors communicated.²¹ While they were at the computer, doctors gave short responses to patients' questions, delayed responding, glanced at the screen rather than looking at the patient, or structured the interview around the computer rather than the patient. On the positive side, recent experiences in Norway have identified benefits to remote telemedical consultation. Services that provided isolated general practitioners with access to specialist skills had an unexpected side effect. The skills of the general practitioners were increased by repeated interactions with specialists during the management of cases that were previously referred.²² This may arise through the dynamics of the relationship between a remote general practitioner and a specialist. Unlike in most educational settings, both are motivated to form a coach and apprentice relationship for the immediate management of a patient.

Thirdly, probably the most important issue for researchers is to understand the effect of introducing technologies that allow asynchronous communication. At present, devices such as telephones and pagers interrupt people when communication is desired--this is synchronous communication. The messages sent across asynchronous systems such as electronic mail and voice mail do not need to be answered immediately and so have the potential to reduce the number of interruptions experienced by clinicians. Such messages may nevertheless carry important information. It will be critical to understand how such systems can be designed to ensure that health care workers do not miss important information and at the same time are not inundated with a flood of irrelevant messages.

Finally, along with new communication possibilities, come new medicolegal implications. In the United States the courts have decided that radiologists are negligent if they fail to inform clinicians personally of a diagnosis. "Communication of an unusual finding in an x ray, so that it may be beneficially utilised, is as important as the finding itself."²³ Furthermore, leaving a message with an intermediary is not enough--"certain medical emergencies may require the most direct and immediate response involving personal consultation and exchange."²³ The fact that such communication requirements are beginning to be mandated reflects the community's changing perceptions of best medical practice.

The rapid arrival of telemedicine suggests that the health care community is beginning to identify the benefits of good clinical communications practice and to realise the costs of poor communication. The next few years should see the research in telemedicine mature. The main focus will become the application of communication technologies rather than their development. This represents the same shift in focus that was required of medical informatics, in which initially much effort was spent in developing technologies specifically for medicine.

Protocol based decision support

Many see the development of protocol based medicine as the essential cultural change in clinical practice that will permit the design of useful clinical information systems.²⁴ It was rightly seen as inappropriate for early computer system designers to try to regularise clinical practice to suit the nature of their systems. The move to evidence based medicine now begins to make it acceptable for clinicians to follow standard assessment and treatment protocols.²⁵ In this case it is quite appropriate for clinicians to use information systems to help them.

View larger version (281K): [in this window] [in a new window]   OncoLink's home page. Reproduced with permission

The ultimate goal of a protocol based decision support system is to provide a set of tools that allows a clinician to access up to date guidelines and then apply them to the management of their patients. Simple protocol systems will probably appear in clinical practice by the end of the decade.²⁶ In some sense, first generation systems have already appeared as treatment guidelines and clinical trial data can now be accessed on the internet (box 1).¹²

Evidence suggests, however, that even when guidelines are available, clinicians forget to follow them or deviate from them without clear cause.²⁶ Forgetting preplanned management tasks seems to be especially likely when making clinical decisions in high stress situations.²⁷ Yet enforcing uniform adherence to guidelines is probably unacceptable, given the complexity of individual cases. It should be possible, however, to make it as easy as possible for clinicians to access guidelines during routine care, making it less likely that steps will be inadvertently forgotten or altered.

This will require the design of more complex systems that will be integrated into the electronic patient record such that protocols can be stored and manipulated by clinicians. For example, best practice recommendations may need to be customised for local conditions or for individual patients. Furthermore, guidelines may be incorporated directly into patient records. As elements of the guideline are completed, they could be automatically noted. The records of care generated in this manner might ultimately be used for population based outcomes analysis.

Some researchers advocate the use of computerised protocols in more complex settings. One group uses a set of ventilation protocols to adjust tidal volume and ventilator rate settings for patients with the adult respiratory distress syndrome.²⁸ They report using the system for over 50000 hours on 150 such patients.²⁹ In one trial with 12 patients, 94% of 4531 protocol generated recommendations were followed by staff. The survival rate of the patients supported with computerised protocols was four times the expected rate from historical controls.³⁰

Two key problems will be faced as such systems become more commonplace. The first is the arduous but essential collation of best practice guidelines, which needs to be carried out by bodies such as the Cochrane Collaboration.¹² In the absence of such collation the value of protocol systems will be minimal. The second concerns issues at the heart of informatics--the problems of defining, managing, and updating medical terminology.

View larger version (126K): [in this window] [in a new window]   Home page of National Library of Medicine's visible human project. Reproduced with permission

Terminological systems

Medical coding systems such as versions of the International Classification of Diseases (ICD), the systematised nomenclature of medicine (SNOMED*RF 30a*), and the Read system*RF 30b* are becoming increasingly familiar to clinicians. Their rationale is as follows. Once captured electronically, clinical data should be available for subsequent aggregation and analysis. However, the words used to describe conditions vary so much that simple analysis is often not possible. Furthermore, the meanings attached to terms may vary. If there was an agreed set of terms to describe the process of care then data analysis would be simplified.³¹ The goal of research into medical terminologies is to arrive at a consensus on the most appropriate set of terms and the way they should be structured.

The fundamental advance in terminological research over the past year or so is the realisation that the goal of constructing a complete and universal thesaurus of medical terms is ill posed. Terminology evolves in a context of use, and attempting to define context independent terminologies is ultimately implausible. Coupled with this view comes the pragmatic understanding that a more robust scientific approach needs to be brought to the enterprise of terminology construction. Each of these issues deserves to be examined in some detail.

Universal terminological systems

The ideal terminological system would be a complete, formal, and universal language that allowed all medical concepts to be described and reasoned about. Some researchers have explicitly asserted that building such a singular and "correct" medical language is their goal.^{32 33} This task emphasises two clear requirements: the ability for the terminological language to cover all the concepts that need to be reasoned about and the independence of the terminology from any particular reasoning task. A further requirement occasionally discussed is that when alternative terminologies exist, they must be logically related such that one can be translated into the other.

Despite the enormous health care investment currently devoted to achieving these goals, current evidence indicates that they are not possible. No set of codes or terms can be universally applied in medicine. There are two fundamental and related obstacles to devising a universal terminological system. The first is the problem of model construction--terminologies are simply a way of modelling the world, and the world is always richer and more complex than any model that humans can devise. The second is the problem of symbol grounding. The words we use to label objects do not necessarily reflect the way we think about the objects, nor do they necessarily reflect defined objects in the real world.³⁴ The cumulative evidence from recent thinking in cognitive science, computer science, and artificial intelligence provides a formidable set of supporting arguments.

Cognitive studies of the way people form categories have shifted from the view that categories exist objectively to the notion that concepts are relative and structured around probabilistic prototypes.³⁵ The qualities of prototypical categories are only generally true of the examples they classify. For example, most people would happily say that flight was a property of birds and be able to cope with the fact that some birds are flightless. The category bird has no pure definition. The way in which people use family resemblances to create such categories from examples remains an area of research.³⁶ Many researchers in artificial intelligence also contend that there is no objective model of medical knowledge. Much of this is based on their experiences in constructing and maintaining knowledge based systems.³⁷

Furthermore, people choose categories at a level of description that is appropriate for thinking about an object in most situations.³⁵ Categories are formed entirely based on their usefulness. Medicine's terminologies have evolved over many years and are also subject to the same process of cognitive evolution. Consequently, disease entities exist for as long as they are useful mental constructs, and are replaced as better concepts emerge--there is no static body of medical knowledge. Not only are new concepts added, but often the very structure of medical knowledge changes as concepts are internally reorganised.^{38 39} The ninth and tenth versions of ICD are substantially different, partly because of the changes in medicine over the 15 years in which the tenth revision was developed.

Any attempt at modelling medical knowledge by the imposition of a structure on its terms will thus decay in accuracy over time.^{41 42} Consequently it does not make sense to think of terminological systems developing independently of a context of use. Even those who seek to build a canonical medical terminology are forced to select a clinical application to set a context before they can meaningfully proceed.⁴³

Equally, there is no reason to expect that there is any uniform mapping between terminological systems developed in different contexts of use.^{44 45} Even when the systems are of similar construction, problems are encountered when knowledge is translated from one form into another. Heinsohn et al concluded that sharing knowledge between terminological systems "does not seem to be easily achievable."⁴⁶

Building maintainable and testable terminologies

Although coding systems can never be truly canonical, they still provide a practical basis for managing the language of medicine--so long as it is understood that they define a limited and consensual language that will have to be continually modified. This modification is a predictable consequence of the subjectivity of knowledge. Whenever a knowledge base is applied to a task outside of its intended use, it will require change.³⁷

The systematised nomenclature of medicine, for example, was initially developed to classify pathological items. It has now been expanded to produce a general purpose system for all of medicine. A study of this nomenclature's utility in coding nursing reports found, however, that it coded only about 69% of terms,⁴⁷ which implies that the missing terms would need to be added. Such additions are required every time a terminology is applied to a new subject, making the task of updating problematic.⁴⁸

Eventually, as a terminology is continually expanded into new subjects, its fundamental organisational structure will be altered to reflect the different structure of the new subjects.³⁸ The process of terminology growth and alteration introduces huge problems of maintenance and the possibility that the system will start to incorporate errors, duplications, and contradictions. If terminological thesauruses are regarded as computer programs then continued modification is a poor strategy for development. Lessons from the field of software engineering suggest that the best time to modify a program is early in the development cycle. Introducing changes into a mature system becomes increasingly expensive over time.⁴⁹ Consequently, we have probably reached the stage when uncontrolled addition of terms to existing thesauruses is no longer acceptable. Those who pay for their maintenance will be faced with ever increasing costs. To manage these costs the performance of a thesaurus on a particular task needs to be measured; then the cost of proposed additions or alterations to improve performance needs to be calculated.

Compositional terminologies

In the longer term new approaches are needed. Most existing coding systems are enumerative, listing all the possible terms that could be used in advance. Such systems are developed independently of each other. A compositional approach uses basic terms as building blocks in conjunction with specialised methods to generate terms for specialised needs. Mapping between specialised terms is not uniformly possible with enumerative systems but is inherent in the design of compositional systems.

The compositional approach, in which terms are created from a more basic set of components, may be more practical to build, maintain, and update.⁴⁵ For example, a practitioner may ask whether severe discomfort in the fifth left metacarpophalangeal joint in a patient record corresponds to small joint symptoms in a clinical protocol. An enumerative system would have to have a pre-existing code for the clinical findings, but a compositional system would generate the findings from a set of components. Indeed, it should be able to generate many such specific conjunctions, so long as they are medically sensible.⁵⁰ Thus rather than developing static terminologies, the combinatorial approach tries to construct dynamic terminology servers to produce answers to a variety of questions.⁵¹

View larger version (173K): [in this window] [in a new window]   Paper from "Journal of Medical Imaging" on internet. Reproduced with permission

There are two hypotheses behind the compositional proposal. The first is an engineering hypothesis--that compositional systems are easier and cheaper to maintain and update than enumerative ones. As we have seen, current enumerative systems continually require extensions that will over time introduce inconsistencies to the system. The compositional approach starts from scratch, defining a core of components that constitute a deep model of medical knowledge. The expectation is that terms can be generated from a compositional model. By definition, since they are generated from the same core of knowledge and the method of generation is known, terms can be mapped on to one another logically. Furthermore, as medicine changes, these changes can be made to the core and be immediately reflected in any new term generated. Compositional systems should also allow the use of sophisticated internal checks on the correctness of their contents.⁵²

Compositional systems should also be more efficient to use. The power of a compositional system is its compactness and maintainability, while the cost of using it is that each answer has to be derived from first principles, and this takes computer time. However, the more expressive and complete an enumerative system is, the slower it is to use.⁴⁶ One of the engineering trade offs to be explored in the future will be to decide whether a compositional system is quicker to interrogate than a larger enumerative one. The evidence from other disciplines is that the compositional approach will eventually be fastest, as enumerative systems grow to be too large. For example, in computer engineering, so called reduced instruction set computer chips (RISC) have a small set of basic operations which can be combined to do more complex operations. These chips are much faster than traditional ones that have a large enumeration of operations to cover many eventualities.

The second compositional hypothesis is a scientific one and is more controversial--that there is such a thing as a deep or core set of medical knowledge from which terms can be generated.⁴³ Like their enumerative counterparts, compositional systems are only models of the world. They are subject to the same issues of model fidelity and subjectivity. Hence there is no greater depth to the knowledge they encode--it is either just more detailed or more general.⁵³

The way forward

In the short term administration agencies keen to obtain aggregate clinical data are driven to adopt existing systems, even if they are imperfect. This has led to much debate among those supporting particular systems of their merits over competing ones.⁴²

Doctors in the United Kingdom have been asked to adopt the version 3 Read codes for use not only in personal clinical systems but also in audit, research, outcomes, and guidelines.⁵⁴ Such a decision can now be seen to be necessarily interim. What is really needed to help rational choices in the longer term is impartial empirical research, comparing the cost and efficacy of different systems in support of well defined tasks and contexts. For example, in a recent study comparing the utility of different coding schemes in classifying problem lists from medical records, none of the major systems was found to be comprehensive. The unified medical language system (UMLS) and the systematised nomenclature of medicine were found to be superior to Read and ICD-9 clinical modifications.⁵⁵

In contrast with the British approach, however, the Board of Directors of the American Medical Informatics Association has suggested that it is not necessary or desirable to have all codes coming from a single master system. It suggests that several existing and tested approaches should be embraced, despite their imperfections, in order to progress quickly. A first phase system could be created by borrowing from the different existing code systems, each created for and therefore better suited to different subject domains.³¹

The longer term need will be to introduce more maintainable and extensible systems as the cost of supporting existing systems becomes insupportable. A solution based in part on multiple compositional systems would seem to be most desirable. Since any general medical terminology will cover only a small part of the specific vocabulary of any medical specialty, separate systems may need to be developed for use between and within specialties--"vocabularies need to be constructed in a manner that preserves the context of each discipline and ensures translation between disciplines."⁵⁶ Indeed over a century ago, when Farr constructed the classification system ultimately resulting in the ICD he noted that "several classifications may, therefore, be used with advantage; and the physician, the pathologist, or the jurist, each from his own point of view, may legitimately classify the diseases and the causes of death in the way that he thinks best adapted to facilitate his enquiries."⁵⁷

Specialised compositional systems will thus need to be constructed that agree on a restricted subset of terms necessary for the passage of information between specialties--a kind of Esperanto between different cultures. Work on such communication standards is at present still in its infancy,⁵⁸ and more substantive work should be expected in the future. Currently, terms are created without explicit tasks in mind, in the hope that all unseen eventualities will be served. Interspecialty systems would probably need to be tightly task based to ensure maximum utility.

At this point the importance of protocol based medicine becomes very clear. Protocols are constructed with an explicit task and context in mind. They are written by experts within a specialty, who arrive at a consensus on the management of a specific condition. In the process of doing so they have to define their terms. The communication of information to another specialty can also be defined in the same manner. Given that a patient is being treated according to a protocol, what information is needed by an allied specialist? Although good terminologies will clearly be needed to construct computer based protocol systems,⁴⁵ the discipline of writing protocols will constrain the terminology problem sufficiently such that a well defined and relevant set of terms can be agreed on.

Conclusion

I have reviewed three apparently quite separate areas--telemedicine, protocol based decision support systems, and terminologies. They can now be seen to be inextricably entwined since the goals of communicating information and deciding on the content of information cannot be separated. Human communication entails information exchange in a context.¹⁹ What is said depends on the intended message, the method used to convey the message, who is speaking, and who is being spoken to. The development of protocol based systems and their supporting terminological systems is a perfect example of that symbiosis.

Appendix

Addresses on the World Wide Web

General medical indexes to internet resources

World Wide Web Virtual Library: Biosciences--Medicine http://golgi.harvard.edu/biopages/medicine.html

Whole Internet Catalog--Health and Medicine http://nearnet.gnn.com/wic/med.toc.html

HealthNet http://hpb 1.hwc.ca/healthnet/#medapp

Internet services mentioned in the text

HospitalWeb http://dem0nmac.mgh.harvard.edu/hospitalweb.html

Interactive Medical Student Lounge http://falcon.cc.ukans.edu:80/~nsween/

Journal of Medical Imaging http://jmi.gdb.org/JMI/ejourn.html

US National Library of Medicine (NLM) http://www.nlm.nih.gov/

OncoLink http://cancer.med.upenn.edu/

Virtual Hospital http://indy.radiology.uiowa.edu/VirtualHospital.html

Visible Human Project http://www.nlm.nih.gov/extramural_research.dir/visible_human.html

WebPath: Internet Pathology Laboratory http://www-medlib.med.utah.edu/WebPath/webpath.html

Cochrane Collaboration http://hiru.mcmaster.ca/cochrane/cochrane.htm

1. Coiera E. Question the assumptions. In: Barahona P, Christensen JP, eds. Knowledge and decisions in health telematics. Amsterdam: IOS Press, 1994:61-6.
2. Wyatt J. Promoting use of medical knowledge systems: lessons from computerised ECG interpreters. In: Barahona P, Christensen JP, eds. Knowledge and decisions in health telematics. Amsterdam: IOS Press, 1994:73-80.
3. Van der Lei J. Computer-based decision support: the unfulfilled promise. In: Barahona P, Christensen JP, eds. Knowledge and decisions in health telematics. Amsterdam: IOC Press, 1994;67-72.
4. Miller RA. Medical diagnostic decision support systems--past, present, and future: a threaded bibliography and brief commentary. Journal of the American Medical Informatics Association 1994;1:8-27. [Abstract/Free Full Text]
5. Hunter L, ed. Artificial intelligence and molecular biology. Menlo Park, CA: AAAI Press/MIT Press, 1993.
6. Keravnou E, ed. Deep models for medical knowledge engineering. Amsterdam: Elsevier Science, 1992.
7. Wyatt J. The evaluation of clinical decision support systems: a discussion of the methodology used in the ACORN project. Proceedings of AIME 87. Lecture Notes in Medical Informatics. Berlin: Springer, 1987;33:15-24.
8. Nymo BJ. Telemedicine. Telektronikk 1993;89(1):4-11.
9. Little AD. Telecommunications: can it help solve America's health care problems? Cambridge, MA: Arthur D Little, 1992.
10. Bowles RA, Teale R. Communications services in support of collaborative health care. BT Technology Journal 1994;12(3):29-44.
11. Buhle EL, Goldwein JW, Benjamin I. OncoLink: a multimedia oncology information resource on the Internet. Proceedings of the Symposium on computer applications in medicine. Journal of the American Medical Informatics Association 1994; symposium suppl 1994;103-7.
12. Goodlee F. The Cochrane Collaboration. BMJ 1994;309:969-70. [Free Full Text]
13. Rao JN. Follow up by telephone. BMJ 1994;309:1527-8. [Free Full Text]
14. Branger PJ, van der Wouden JC, Schudel BR, Verboog E, Duisterhout JS, et al. Electronic communication between providers of primary and secondary care. BMJ 1992;305:1068-70.
15. Fitzpatrick K, Vineski E. The role of cordless phones in improving patient care. Physician Assistant June 1993:87-92.
16. Constable J. Active voice. BJ Healthcare Computing and Information Management 1994;11:30-1.
17. Leirer VO, Morrow DG, Tanke ED, Pariante GM. Elders' nonadherence: its assessment and medication reminding by voice mail. Gerontologist 1991;31:514-20. [Abstract]
18. Fridsma DB, Ford P, Altman R. A survey of patient access to electronic mail: attitudes, barriers and opportunities. Proceedings of the symposium on computer applications in medicine. Journal of the American Medical Informatics Association 1994; symposium suppl: 15-9.
19. McCarthy JC, Monk F. Channels, conversation, co-operation and relevance: all you wanted to know about communication but were afraid to ask. Collaborative Computing 1994;1:35-60.
20. Stoupa R, Campbell J. Documentation of ambulatory care rendered by telephone: use of a computerized nursing module. In: Miller R, ed. Proceedings of the symposium on computer applications in medicine. Los Alamitos, CA: IEEE Computer Society Press, 1990:890-3.
21. Greatbach D, Luff P, Heath C, Campion P. Interpersonal communication and human-computer interaction: an examination of the use of computers in medical consultations. Interacting with Computers 1993;5:193-216.
22. Akelsen S, Lillehaug S. Teaching and learning aspects of remote medical consultation. Telektronikk 1993;89:42-7.
23. Kline TJ, Kline TS. Radiologists, communication and resolution. V. A medicolegal issue. Radiology 1992;184:131-4. [Abstract]
24. Durinck J, Coiera E, Baud R, et al. The role of knowledge based systems in clinical practice. In: Barahona P, Christensen JP, eds. Knowledge and decisions in health telematics--the next decade. Amsterdam: IOS Press, 1994:199-203.
25. Mulrow CD. Rationale for systematic reviews. BMJ 1994;309:597-9. [Free Full Text]
26. Renaud-Salis JL. Distributed clinical management-information systems: an enabling technology for future health care programmes. In: Barahona P, Christensen JP, eds. Knowledge and decisions in health telematics--the next decade. Amsterdam: IOS Press, 1994;139-46.
27. Coiera E, Tombs V, Higgins G, Clutton-Brock TH. Real-time clinical decision making. HP Laboratories Technical Report 1994; No 79. (HPL-94-79.)
28. Thomsen GE, Pope D, East T, Morris A, Tupper Kinder A, Carlsom D, et al. Clinical performance of a rule-based decision support system for mechanical ventilation of ARDS patients. In: Safran C, ed. Proceedings of the symposium on computer applications in medicine. New York: McGraw Hill, 1993:339-43.
29. Morris A, East T, Wallace C, Orme J, Clemmer T, Weaver L, et al. Ethical implications of standardization of ICU care with computerized protocols. Proceedings of the symposium on computer applications in medicine. Journal of the American Medical Informatics Association 1994; symposium suppl: 501-5.
30. Morris A, Wallace C, Menlove R, Clemmer T, Orme J, Weaver L, et al. A randomised clinical trial of pressure-controlled inverse ratio ventilation and extracorporeal CO(sub 2) removal from ARDS. American Journal of Respiratory and Critical Care Medicine 1994;149:295-305. [Abstract]

[Medline]

[Medline]
31. Ackerman M, Ball M, Clayton PD, Frisse ME, Gardner RM, et al. Standards for medical identifiers, codes and messages needed to create an efficient computer-stored medical record. Journal of the American Medical Informatics Association 1994;1:1-7. [Abstract/Free Full Text]
32. Cimino JJ. Controlled medical vocabulary construction: methods from the CANON group. Journal of the American Medical Informatics Association 1994;1:296-7. [Free Full Text]
33. Evans D, Cimino JJ, Hersh WR, Huff SM, Bell DS. Toward a medical-concept representational language. Journal of the American Medical Informatics Association 1994;1:207-17. [Abstract/Free Full Text]
34. Norman DA. Cognition in the head and in the world: an introduction to the special issue on situated action. Cognitive Science 1993;17(1):1-6.
35. Rosch E. Principles of categorization. In: Rosch E, Lloyd BB, eds. Readings in cognitive science. Los Altos, CA: Morgan Kaufmann, 1988:312-22.
36. Ahn W, Medin DL. A two-stage model of category construction. Cognitive Science 1992;16:81-121.
37. Clancy WJ. Notes on "Heuristic classification." Artificial Intelligence 1993;59:191-6.
38. Clancy WJ. Notes on "Epistemology of a rule-based expert system." Artificial Intelligence 1993;59:197-204.
39. Laporte R. Patterns of disease: diabetes mellitus and the rest. BMJ 1995;310:545-6. [Free Full Text]
40. NHS Centre for Coding and Classification. A guide to the use of tables of equivalence between ICD-9 and ICD-10. Loughborough: NHSCCC, 1995. (Report F6110.)
41. Hogarth RM. Generalisation in decision research: the role of formal models. IEEE Transactions on Systems Man and Cybernetics 1986;16:439-49.
42. Tuttle S, Nelson SJ. The role of the UMLS in "storing" and "sharing" across systems. Int J Biomed Comput 1994;34:207-37. [Medline]
43. Friedman C, Huff SM, Hersh WR, Pattison-Gordon JJ, Cimino JJ. The CANON Group's effort: working toward a merged model. Journal of the American Medical Informatics Association 1995;2:4-18. [Abstract/Free Full Text]
44. Tuttle MS. The position of the CANON group: a reality check. Journal of the American Medical Informatics Association 1994;1:298-9. [Free Full Text]
45. Glowinski AJ. Integrating guidelines and the clinical record: the role of semantically constrained terminologies. In: Gordon C, Christensen JP, eds. Health telematics for clinical guidelines and protocols. Amsterdam: IOS Press, 1994.
46. Heinsohn J, Kudenko D, Nobel B, Profitlich H. An empirical analysis of terminological representation systems. Artificial Intelligence 1994;68:367-97.
47. Henry S, Campbell K, Holzemer W. Representation of nursing terms for the description of patient problems using SNOMED III. Proceedings of the symposium on computer applications in medicine. New York: McGraw Hill, 1993:700-4.
48. Cimino JJ, Clayton PD. Coping with changing controlled vocabularies. Proceedings of the symposium on computer applications in medicine. Journal of the American Medical Informatics Association 1994; symposium suppl: 135-9.
49. Littlewood B, ed. Software reliability--achievement and assessment. Oxford: Blackwell Scientific Publications, 1987.
50. Glowinski AJ, Coiera EW, O'Neil MJ. The role of domain models in maintaining the consistency of large medical knowledge bases. Proceedings of the third AIME. Lecture Notes in Medical Informatics 1991;44:72-81.
51. Nowlan WA, Rector AL, Rush TW, Solomon WD. From terminology to terminology services. Proceedings of the Symposium on Computer Applications in Medicine. Journal of the American Medical Informatics Association 1994; symposium suppl: 150-4.
52. Goble CA, Glowinski AJ, Nowlan A, Rector A. A descriptive semantic formalism for medicine. Proceedings of the ninth IEEE international conference on data engineering. Los Alamitos, CA: IEEE Press, 1993: 624-32.
53. Coiera E. Intermediate depth representations. Artificial Intelligence in Medicine 1992;4:431-45.
54. Calman KC. New national thesaurus. CMO's Update 1994;4:1.
55. Campbell JR, Payne TH. A comparison of four schemes for codification of problem lists. Proceedings of the symposium on computer applications in medicine. Journal of the American Medical Informatics Association 1994; symposium suppl: 201-4.
56. Brennan F. On the relevance of discipline in informatics. Journal of the American Medical Informatics Association 1994;1:200-1. [Free Full Text]
57. World Health Organisation. Manual of the international statistical classification of diseases, injuries and causes of death. Geneva: WHO, 1977.
58. Ma H. Mapping clause of Arden syntax with the HL7 and ASTM E 1238-88 standard. Int J Biomed Comput 1995;38:9-21. [Medline]