BMJ  2004;328:1162 (15 May), doi:10.1136/bmj.328.7449.1162

Paper

Doctors' experience with handheld computers in clinical practice: qualitative study

¹ Division of Health Services Management and Policy, Ohio State University, School of Public Health, 1583 Perry Street, Atwell 246, Columbus, OH 43210-1234, USA, ² Division of General Internal Medicine, Ohio State University College of Medicine and Public Health, Columbus, OH 43210, USA

Abstract

Objective To examine doctors' perspectives about their experiences with handheld computers in clinical practice.

Design Qualitative study of eight focus groups consisting of doctors with diverse training and practice patterns.

Setting Six practice settings across the United States and two additional focus group sessions held at a national meeting of general internists.

Participants 54 doctors who did or did not use handheld computers.

Results Doctors who used handheld computers in clinical practice seemed generally satisfied with them and reported diverse patterns of use. Users perceived that the devices helped them increase productivity and improve patient care. Barriers to use concerned the device itself and personal and perceptual constraints, with perceptual factors such as comfort with technology, preference for paper, and the impression that the devices are not easy to use somewhat difficult to overcome. Participants suggested that organisations can help promote handheld computers by providing advice on purchase, usage, training, and user support. Participants expressed concern about reliability and security of the device but were particularly concerned about dependency on the device and over-reliance as a substitute for clinical thinking.

Conclusions Doctors expect handheld computers to become more useful, and most seem interested in leveraging (getting the most value from) their use. Key opportunities with handheld computers included their use as a stepping stone to build doctors' comfort with other information technology and ehealth initiatives and providing point of care support that helps improve patient care.

Introduction

A quarter of US doctors use handheld computers (personal digital assistants), with the proportion predicted to double by 2005.¹ Many uses have been found for these devices in clinical practice, but little research has examined how doctors view their experience with them, especially the effects on patient care. We studied doctors' experiences with and perceptions of handheld computers in various clinical settings.

Methods

Our findings come from doctors' focus groups as one component of a two part qualitative study designed to explore experiences with handheld computers in clinical practice from the perspectives of both organisations and doctors.² Three broad issues guiding our groups were: how and why doctors use handheld computers in clinical practice, what barriers doctors perceive with their use and how these could be overcome, and what doctors expect from their use in the future. Issues that emerged during the study were also explored. We purposely sampled organisations that were reportedly using handheld computers in clinical practice, and we contacted their affiliated doctors to participate in our study.

Focus groups
We held eight focus group sessions lasting 60-90 minutes between April 2002 and September 2003. Six sessions were conducted at a medical centre, a children's hospital, an independent practice association, two community based healthcare systems, and a community hospital. Two sessions were conducted at a meeting of general internists.

Topics covered included general use of handheld computers, rationale for use, expectations, barriers or challenges, and organisational support. Ongoing analyses led to the inclusion of new topics. These included perceived benefits of handheld computers, behaviour changes with use, and concerns. We asked participants about the specifics of their use of handheld computers and any organisational expectations for their use.

Two study investigators conducted each focus group, facilitating discussion with an open ended list of questions, including probes for more detailed information. The sessions were audio-taped, transcribed, and then verified and corrected by one investigator (ASM, SBS, or MAM).

Participants
Our eight focus groups consisted of 54 doctors. One third of the doctors were women and three quarters were generalists. Nearly half of the participants practised full time, and the remainder were residents or fellows. The groups contained doctors who did or did not use handheld computers, with users representing a variety of levels and patterns of use. One third of participants were affiliated with an academic medical centre and the remainder were affiliated with an independent practice association, community hospital, or children's hospital.

Analyses
We analysed our data using a combination of deductive and inductive methods.³ The investigators read the transcripts, identified broad themes, and discussed emergent findings. This iterative process allowed new ideas and themes to be presented in subsequent sessions. By the end of the sixth session no new major themes emerged, indicating near saturation.⁴ The last two sessions completed the deductive part of our analysis.

The investigators read the transcripts several times and constructed a preliminary coding frame. This frame was applied to two common transcripts, which enabled decisions on coding to be compared and codes to be clarified. We then identified categories and constructed major themes. Periodic discussions among the investigators ensured consistency of coding and helped us reach agreement on final themes. An ongoing review of the literature helped us to compare, validate, and extend our findings.⁴

Results

How doctors use handheld computers in clinical practice
The use of handheld computers varies widely in clinical practice (box 1). Clinicians use administrative functions for the development and sharing of lists and databases to keep track of drug formularies, call schedules, and contact details. Specific applications allow patients to be tracked and clinical results to be monitored. The use of administrative functions linked to clinical activities is expanding, with applications such as electronic prescribing and coding attracting attention because of their potential to increase doctors' productivity.

Participants typically used handheld computers at their own initiation, buying devices based on personal preferences or recommendations. An increasing number of organisations are promoting handheld computers as part of the strategy for clinical information technology, with many academic medical centres purchasing devices for their residents. In one institution, doctors use handheld computers to access patients' electronic records through a browser based application, similar to that in a recent study.⁵ At two organisations we studied, residents use their devices to share patient details between shifts.

Patterns of use and characteristics of handheld computer users
The frequency and intensity of use of handheld computers varied, and on the basis of these we were able to develop user categories (table). Non-users had never used handheld computers or had abandoned them. Niche users included those whose use was restricted to a single application but reported that this limited functionality was sufficiently valuable such that they would continue use. Routine users had integrated handheld computers into their clinical workflow, using multiple applications on a regular basis. Power users were self described "technophiles" who were eager to showcase their latest device.

View this table: [in this window] [in a new window]   Patterns and characteristics of users of handheld computers

 

Box 1: Examples of common uses of handheld computers Point of care assistance—drug information, clinical guidelines, decision aids, patient education Patient information—patient tracking, clinical results Administrative functions—electronic prescribing, coding, tracking schedules Research activities—data collection, participant education Medical education—lecture notes, presentations, photographs and diagrams

Perceived benefits of handheld computers
Users seemed generally satisfied with their handheld computers and perceived many benefits (box 2). We anticipated discussion about productivity gains and convenience, but we also heard many doctors describing how they do things differently and "better" owing to the device. We explored this theme further. One doctor explained "I don't guess that something is not interacting with warfarin and cross my fingers and hope. That's my biggest thing, I don't guess. Or say I will look that up later and not get to it." Similarly, many participants noted how they "look things up more, medication-wise."

Across all groups, users reported that handheld computer applications often provided complementary benefits, improving productivity and interactions with patients: "It reminds me to do things that I might forget to do. Not just be at this meeting, but I can get a glance and see that I haven't done the stool occult blood on that patient because they are in front of me." These findings are supported by other studies.^{6 7}

Box 2: Doctor perceived benefits of handheld computers Enhanced productivity It really saves you the aggravation of looking for something and not being able to find it that minute Anything where you don't have to wait for somebody to finish at the terminal and wait in a long line of doctors who don't have handheld devices... You've got your handheld device, you put your orders in and walk away while the other guy is still waiting. You've got an advantage I feel like it saves me time so I don't have to step out of a room and look something up For me, to be able to sync my Palm before I make rounds and have all that information with me, then I don't have to run around and ask the nurse who says, "I'm not a nurse, I'm a respiratory therapist" Enhanced quality of patient care and service You get a phone call from somebody. You have a clue. Right here, right now, right this minute, I have a clue For example, if you were talking to a patient and you came across a medication that you didn't know, if you didn't have a Palm you probably wouldn't go in the other room and go to the PDR and look it up. But if you have the Palm you call pull it out and say, oh yeah, that is a hypertensive medication I think the way to approach it is quality, and the service you are offering, and the timeliness of the information

Barriers to use of handheld computers
The two main barriers to using handheld computers were personal issues and the device itself. Issues concerning the device included size, limited memory and battery life, and speed of data exchange.^8-12 Many participants expressed frustration, especially with data entry: "You know, with the Palm you are trying to write graffiti. And, you know me, I'm going `Oh, that's wrong.' I can't remember what is a seven? It's coming out two! I think the data entry is real tough."

Two major personal barriers described by non-users were physical constraints, such as eyesight, and perceptual constraints, including comfort with the device and personal preferences (box 3). In contrast, users rarely reported personal barriers and instead described those device features perceived as problematic by non-users as strengths, such as raving about portability rather than complaining about a small screen. Similarly, although non-users reported that "these things just have to be easier," routine and power users described the operating systems as "intuitive."

A major barrier for non-users in all groups was their perception that they did not receive, or expect to receive, enough value from the devices to change their existing patterns of practice. As a former user explained, "it just takes too long and is too disruptive to the day." Another participant said "A lot of residents would open it up and try to load up all the stuff that they need and they would just get so frustrated and didn't know how to do it, that it was taking too much time, that it just wasn't worth it." Furthermore, when computers were readily available, both non-users and former users perceived it as unnecessary to incorporate handheld computers into clinical workflow. Routine and power users, however, provided numerous examples of where devices added value and improved their work routines.

Strategies proposed by doctors to overcome barriers
Participants suggested several ways in which organisations could help to overcome barriers (box 4). Doctors who had never used handheld computers noted that advice about which device to use might be sufficient to tip them into a user category: "For people who aren't used to using computers, it's just not worth the time to figure all that stuff out." Niche users wanted specific advice about the appropriateness of applications, and our impression was that this might shift them towards routine use.

Organisations could provide training and retraining to overcome many barriers. Both niche users and routine users were aware that there was more they could do with the device, if only they knew what and how: "I don't know how to use it to its fullest potential. It's my fault rather than the machine's fault. But I haven't been educated enough to use it to its fullest potential." Participants suggested that training should be available one on one and should involve another clinician (nurse or doctor). With the exception of power users, most users wanted support available constantly from a help desk or expert: "So I guess what we're saying is that maybe there should be like a first aid station... somewhere I can go to, sit down and say this is what I'm trying to do. Why can't I do it? What did I do wrong? How can you help me make it right?"

Doctors reported that the more computer friendly their organisation, the more likely they were to keep using or trying to use handheld computers. A few participants noted that an organisation sensitive to their needs would be an appealing place to practise.

Concerns about handheld computers
Concerns raised in early focus groups were purposely explored further in subsequent groups. We categorised concerns into four areas: the device itself, information security, over-reliance, and potential changes to practice.

Doctors' concerns about the device included loss, breakage, and reliability. Less common were concerns about security. Although doctors expressed concern about secure patient data, they seemed unconvinced that handheld computers represented a greater threat than paper records. In most sessions, satisfaction with convenience seemed to compete with concern about security: "You can lock it but I never do. It is a pain to get in." Power users, however, reported reliably adopting security routines.

A major concern that emerged in every focus group was dependency, particularly among routine users and power users (box 5): "The Palm runs my life—if I lost it! Ugh." Many users also

Box 3: Personal factors creating barriers to handheld computer use Physical constraints Physical factors My fingers are just too big for those buttons I think it will get worse as we start to pull in legacy systems results, and more and more with wireless. I'm not going to be able to see. And I doubt that people without 20/20 vision will be able to read this when we start pulling in information from everywhere Age The problem is that it is hard for me to carry it around. So, I forget it all the time or I don't utilize it and I'm getting old To what extent do people our age actually need to do it? So, if the records aren't all automated and it's a pain in the ass for you, and you've got your list, you know Perceptual constraints Comfort with technology I've talked to a lot of people who have been really disappointed and I think it's just because of lack of experience with computers, and they don't feel comfortable If it doesn't work right, the first time or the second time, it's over Comfort with device But they don't fit in shirt pockets. This thing is just the wrong size for shirt pockets. It fits in pants' pockets. But it goes on and off... so I'm sitting here clicking all day long and wondering what's wrong with my heart valve I found it was cumbersome. I just wasn't really comfortable with it. Heavy. I carried in my suit pocket and was uncomfortable. I carried it in my lab coat and it was uncomfortable Perception that device is not easy to use My partner tried to get it synced, took it home, tried to get it to work the first night. Couldn't do it, quit Preference for paper If you take notes, it's much more practical to take notes on a paper printout and keep your to-do list on that than it is on a palm People like to be able to annotate, they like to have paper. It's tangible. You know, I can write on there "check Mrs. Jones' second potassium," and I can hand that off to someone and they'll do it for me and I can check off if I need to do something. It is very hard to annotate stuff on the Palm yet, I think Preference for personal computers to access information I used it, but I have not found it convenient enough to go and buy one. Where I work we have computers everywhere and I prefer using a keyboard. I have not gotten used to using graffiti. The real estate is so limited on a Palm raised the theoretical concern about becoming over-reliant on the device as a "peripheral brain."

Box 4: Organisational strategies to overcome barriers to use of handheld computers Assistance with selection and set-up Make it ready to go out of the box: Set up PDAs [personal digital assistants] with software and make them ready to sync with a PC [personal computer] I mean that's what a lot of doctors want. They want to just turn it on, start using it. They don't want to have to mess with any of that It would be helpful to know what is best. Is there something already out there or is it better to start from scratch and create your own? Training and retraining But you know orientations just stink. Too much information... you know, we had our palms for like half a day when we had that session. So you hadn't really gotten a chance to do anything with it or look through everything at all I guess personally, I would want instructions Local expert or help desk It's gotta be something where you can go back and dialogue with people and say I'm having a problem here or I'm not getting the full advantage of this thing And you know, when it stops working for some reason, there has to be someone who can do it Handheld computer friendly environment It would be nice to have more sync cradles too I think, rather than having to walk all the way over to the chief's office I think once we get the wireless then that could really save time

Some doctors were concerned about handheld computers changing clinical practice for the worse. Several doctors were concerned that avid users might continuously collect data without furthering patient care. Others were troubled that patients might look negatively upon them for using the devices. A few respondents in each group remarked that they purposely did not use the devices in front of patients, but others were comfortable doing so: "Initially I was afraid that if I had to use a device, they would think I was stupid. But they don't. It doesn't seem that they feel that way. I think I get credit for having a device, which is trendy. So they think I'm smarter." Another group of doctors voiced concern that these devices may become a tool of administration to further constrain their practice, for example, by enforcing guidelines.

Box 5: Doctors' concerns about dependency on handheld computers Dependency I have had them crash before. I don't have another place to look up medications. So I get very dependent on it I am chairman of medicine, director of medical intensive care. I frankly break out in a cold sweat when I lose my Palm. I do. The residents know If I lose this, I get very, very nervous I was surprised how dependent upon it I've become... it's very insidious Over-reliance on device I think.. .there is a part of me that worries that if I become too reliant on the calculator to do all my calculations for me that I will get to the point where I don't have to really remember any formulas and how you look at pH One thing I'm worried about is the fact that I think more and more people are using the Palm as a crutch to a certain extent. And I do the same thing Certain skills you just lose by disuse, basically. Certain things would go away. I mean you would not need to remember. If it's a little esoteric from your mundane use, you will not need to remember it. You won't even try to remember it

Expectations about future use of handheld computers
Most doctors thought that the trend towards incorporating new electronic technologies into medicine would continue. Participants remarked that new doctors were more comfortable with electronic technologies, and this may help promote the use of handheld computers: "The residents coming out right now aren't at that stage right now, but very soon. Not far behind is a group that will only know computers." The contrasting viewpoint, however, was also expressed: "So the question is, how fast does everybody have to change?... A lot of people are going to get away with not learning."

Participants in each group said that handheld computers were destined to become critical because of their potential to improve patient safety and the quality of care. Even when faced with sacrificing personal autonomy, the view for the future was that it is more important to be right and safe (box 6):

So in a lot of ways, our world has been our personal autonomy at getting things right. And more and more that paradigm is moving away. And the requirement of precision is much greater. So I'm not really allowed anymore to get the drug interactions wrong. So, I have to have a device that makes it right... So, if you're going to be held to that standard, then you have to have the tools to be held to that standard. So we're talking about standard of care now, which affects all of us. So, whether you're in medical school and everyone has their Palm Pilot and they're whizzes at it, as opposed to somebody like me who's struggling and wants learn to be able to access and to benefit from this technology, we have to do it... writing illegibly is not going to do it anymore

Although many users were enthusiastic about the potential for handheld computers, most maintained a sense of balance in their perspective: "Just like anything else, it's a tool. It's not the end all be all. It has its pros and its cons, and you just have to learn to get used to it. In some ways it's made our lives easier, and in others it's a bit more cumbersome."

Discussion

Doctors seem to expect handheld computers to become increasingly useful, if not ubiquitous. Organisations can help doctors leverage the use of devices in several ways. Firstly, they can develop applications to facilitate the downloading of material otherwise available on paper, such as databases, drug formularies, and schedule information, but organisations must ensure that these resources are accurate or they will be promptly abandoned. Secondly, organisations can provide advice, training, and user support and create opportunities for doctors to learn from each other. Finally, they can develop options for mobile access to essential point of care information that can be used on handheld computers.

Box 6: Expectations for change in the future And I think what it all boils down to is just the time. It takes too long to manually enter everything. And... you know, if it could do it automatically, I think everybody would do it Once we go to electronic medical records and vital signs and everything is entered in there and you can be wireless and get everything... You wouldn't need to get the cardex, you wouldn't need to get the vital signs, if all that is electronic, you could get that off the PDA I think every day in medicine there is more stuff that you gotta know and things are more complex. I think electronics is going to be our saviour for our sanity and for medical errors and all that kind of stuff. I think there just has to be a place to deposit data and retrieve it fast. I think it is just inevitable. I think more and more of these requirements are going to come down the pike and everyone is going to have to rely on them. Think about when you have got to screen people who have DVTs. What the hell do you do for somebody who has their first DVT? What things should you order and all that kind of stuff. I hope it is on PDA pretty soon

What is already known on this topic Little research has examined doctors' perspectives about experience with handheld computers It is not understood how doctors across practice settings view or value the devices, nor if they have concerns What this study adds Doctors who use handheld computers seem satisfied with them and perceive that they enhance patient care; they expect devices to be more useful in the future when input becomes easier and when organisations offer options for wireless connectivity Concerns about the devices include reliability and dependency Organisations could promote devices by providing training, user support, and advice to build confidence in the technology and its capabilities; organisations can also leverage use by developing handheld formatted databases and options for mobile access to essential point of care information

Our sample was limited to the United States. Although we sought representation from users and non-users of handheld computers, participants were self selecting, and participation was voluntary. It is likely that doctors interested in handheld computers would be more inclined to participate, although our groups did include sceptics and non-users.

As the use of handheld computers in clinical practice is relatively new, and because organisational use of handheld computers varies widely,¹³ we included doctors at various stages in both their own and their organisation's learning about the devices. Our findings are therefore in part influenced by each doctor's place on the learning curve.

Developing strategies to accommodate handheld computers in clinical practice may be advantageous for both institutions and doctors, especially when the devices are used to access clinical information systems, promoting both enhanced safety and improved time efficiency for doctors. When the expected benefits of electronic health records and other electronic applications largely depend on doctors' use of technology, strategies to promote use of such technologies are critical. For many doctors, handheld computers are emerging as a key means to develop familiarity with and to access electronic clinical information. These devices thus may serve as a technology stepping stone for doctors as they face new ehealth initiatives.

---

Contributors: ASMcA, SBS, and MAM were responsible for the study design, data collection, analysis, data interpretation, writing, and submission of this manuscript; they will act as guarantors for the paper. The guarantors accept full responsibility for the conduct of the study, had access to the data, and controlled the decision to publish. ASMcA led the writing and preparation of the manuscript for publication.

Funding: Center for Health Management Research.

Competing interests: None declared.

Ethical approval: Behavioral and Social Sciences Institutional Review Board, Ohio State University (IRB protocol number 01B012).

References

1. Harris Interactive. Doctors' use of handheld personal computing devices increases from 15% in 1999 to 26% in 2001. www.harrisinteractive.com/news/allnewsbydate.asp?NewsID=345 (accessed 1 October 2003).
2. Maxwell J. Qualitative research design. Thousand Oaks, CA: Sage, 1996.
3. Miles M, Huberman A. Qualitative data analysis. Thousand Oaks, CA: Sage, 1994.
4. Glaser B, Strauss A. The discovery of grounded theory: strategies for qualitative research. New York: Aldine de Gruyter, 1967.
5. Chen E, Medonca E, McKnight L, Stetson P, Lei J, Cimino J. PalmCIS: a wireless handheld application for satisfying clinical information needs. J Am Med Inform Assoc 2004;11: 19-28.[Abstract/Free Full Text]
6. Doolan D, Bates D, James B. The use of computers for clinical care: a case series of advanced US sites. J Am Med Inform Assoc 2003;10: 94-107.[Free Full Text]
7. Rothschild J, Lee T, Bae T, Bates D. Clinician use of a palmtop drug reference guide. J Am Med Inform Assoc 2002;9: 223-9.[Abstract/Free Full Text]
8. Fischer S, Stewart T, Mehta S, Wax R, Lapinsky S. Handheld computing in medicine. J Am Med Inform Assoc 2003;10: 139-49.[Abstract/Free Full Text]
9. Schneider S, Kostecke R, Tokazewski J. Buying your first PDA. Fam Pract Manag 2001;8: 50-1.
10. Embi P. Information at hand: using handheld computers in medicine. Cleve Clin J Med 2001;68: 840-9.[ISI][Medline]
11. Wright P, Bartram C, Rogers N, Emslie H, Evans J, Wilson B, et al. Text entry on handheld computers by older users. Ergonomics 2000;43: 702-16.[Medline]
12. Wilderman I, Dobrousin A, Cameron S. Which handheld computer is better for doctors? Part 1: comparing models with Palm operating systems. Can Fam Doctor 2003;49: 1507-8, 1510-1.
13. Al-Ubaydli M. Handheld computers for doctors. West Sussex, England: John Wiley, 2003.

(Accepted 13 April 2004)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Handheld computers in clinical practice: ... and will appeal to new breed of general practitioners

David A Eso
BMJ 2004 328: 1565. [Extract] [Full Text]

Handheld computers may improve patients' care
BMJ 2004 328: 0. [Full Text]

This article has been cited by other articles:

• Yu, F., Houston, T. K., Ray, M. N., Garner, D. Q., Berner, E. S. (2007). Patterns of Use of Handheld Clinical Decision Support Tools in the Clinical Setting. Med Decis Making 27: 744-753 [Abstract]  
• Yarbrough, A. K., Smith, T. B. (2007). Technology Acceptance among Physicians: A New Take on TAM. Med Care Res Rev 64: 650-672 [Abstract]  
• Fahy, B. G., Ketzler, J. T. (2007). Implementation of a Handheld Electronic Point-of-Care Billing System Improved Efficiency in the Critical Care Unit. J Intensive Care Med 22: 374-380 [Abstract]  
• Hauser, S. E., Demner-Fushman, D., Jacobs, J. L., Humphrey, S. M., Ford, G., Thoma, G. R. (2007). Using Wireless Handheld Computers to Seek Information at the Point of Care: An Evaluation by Clinicians. J. Am. Med. Inform. Assoc. 14: 807-815 [Abstract] [Full text]  
• MacGregor, D. L., Tallett, S., MacMillan, S., Gerber, R., O'Brodovich, H. (2006). Clinical and Education Workload Measurements Using Personal Digital Assistant-Based Software. Pediatrics 118: e985-e991 [Abstract] [Full text]  
• Ray, M. N., Houston, T. K., Yu, F. B., Menachemi, N., Maisiak, R. S., Allison, J. J., Berner, E. S. (2006). Development and Testing of a Scale to Assess Physician Attitudes about Handheld Computers with Decision Support. J. Am. Med. Inform. Assoc. 13: 567-572 [Abstract] [Full text]  
• McAlearney, A. S., Schweikhart, S. B., Medow, M. A. (2005). Organizational and Physician Perspectives about Facilitating Handheld Computer Use in Clinical Practice: Results of a Cross-Site Qualitative Study. J. Am. Med. Inform. Assoc. 12: 568-575 [Abstract] [Full text]  
• Horsley, A, Forster, L (2005). Handheld computers in medicine: the way forward. Postgrad. Med. J. 81: 481-482 [Abstract] [Full text]  
• Eso, D. A (2004). Handheld computers in clinical practice: ... and will appeal to new breed of general practitioners. BMJ 328: 1565-1565 [Full text]  

Rapid Responses:

Read all Rapid Responses

Handheld Computer in Clinical Practice

David. A. Eso
bmj.com, 16 Jun 2004 [Full text]