BMJ 1999;319:32-36 ( 3 July )

Information in practice

Computer support for recording and interpreting family histories of breast and ovarian cancer in primary care (RAGs): qualitative evaluation with simulated patients

Jon Emery, Cancer Research Campaign primary care oncology research fellow a,  Robert Walton, senior research fellow a,  Andrew Coulson, research fellow b,  David Glasspool, research fellow b,  Sue Ziebland, senior research fellow a,  John Fox, head b. 

a ICRF General Practice Research Group, Division of Public Health and Primary Health Care, Institute of Health Sciences, Oxford OX3 7LF, b ICRF Advanced Computation Laboratory, PO Box 123, London WC2A 3PX

Correspondence to: J Emery jon.emery{at}green.ox.ac.uk

    Abstract
Top
Abstract
Introduction
Participants and methods
Results
Discussion
References

Objectives: To explore general practitioners' attitudes towards and use of a computer program for assessing genetic risk of cancer in primary care.
Design: Qualitative analysis of semistructured interviews and video recordings of simulated consultations.
Participants: Purposive sample of 15 general practitioners covering a range of computer literacy, interest in genetics, age, and sex.
Interventions: Each doctor used the program in two consultations in which an actor played a woman concerned about her family history of cancer. Consultations were videotaped and followed by interviews with the video as a prompt to questioning.
Main outcome measures: Use of computer program in the consultation.
Results: The program was viewed as an appropriate application of information technology because of the complexity of cancer genetics and a sense of "guideline chaos" in primary care. Doctors found the program easy to use, but it often affected their control of the consultation. They needed to balance their desire to share the computer screen with the patient, driven by their concerns about the effect of the computer on doctor-patient communication, against the risk of premature disclosure of bad news.
Conclusions: This computer program could provide the necessary support to assist assessment of genetic risk of cancer in primary care. The potential impact of computer software on the consultation should not be underestimated. This study highlights the need for careful evaluation when developing medical information systems.


Key messages

  • General practitioners are under increasing pressure to advise their patients about genetic predisposition to various diseases

  • Computers could help doctors to give genetic advice by simplifying the construction and assessment of family trees and implementing referral guidelines

  • This qualitative evaluation explored the context in which a computer program for assessing genetic risk of cancer would be used in general practice and issues surrounding its integration into a consultation

  • Most of the doctors found the program easy to use, but it affected their control of the consultation---because of their desire to share the computer screen with the patient and their inability to anticipate the information that would be displayed

  • The study identified important issues relating to the use of computers in consultations which may be of use in testing software for primary care in the future



    Introduction
Top
Abstract
Introduction
Participants and methods
Results
Discussion
References

Primary care will inevitably play an increasing role in genetics because of rapid advances in genetic medicine and resultant pressures on specialist services.1 The availability of tests for genetic predisposition to breast and colon cancer has resulted in increased referrals to genetic clinics, although many of these people are at low risk.2 Most general practitioners lack the skills and knowledge required to provide a first line genetics service, particularly for multifactorial diseases.3 Computers could help doctors by simplifying the construction and assessment of pedigrees and implementing referral guidelines. 4 5 Although programs exist for drawing pedigrees and assessing risk, they are too complex for primary care and do not provide advice about patient management. 6 7

Computers increase the use of guidelines 8 9 and can improve clinical performance.10 Despite this, decision support has not been adopted in routine general practice. This partly reflects limited clinical input in the development of medical expert systems.11 The selective use of quantitative approaches to evaluate medical information systems has been criticised,12 and multiple methods have been proposed that examine professional and organisational factors influencing use of information systems.13


Clinical scenarios used in simulated consultations

Case 1

Pam, aged 30, has come to see her general practitioner because her mother has recently been diagnosed with breast cancer again. Her mother first developed breast cancer when she was 56. At a follow up clinic six months later, a lump in the other breast was found, which has also turned out to be malignant. She is due to go in for surgery next week. Pam's maternal aunt, Jane, developed ovarian cancer aged 50 and died nine months later. Pam's maternal grandmother died of breast cancer aged 64. Pam doesn't know much about this and doesn't remember her well since she was quite young at the time. Pam's mother is worried that her daughter will get cancer and has told Pam to see her doctor to see if anything can be done. Pam is not particularly concerned, and the issue has been discussed only recently when her mother picked up a leaflet at the hospital about cancer running in families. Pam is married and has no children. She resembles her aunt Jane physically and wonders whether this is important.

  Estimate of risk by Cyrillic program (version 2.1): heterozygote risk for BRCA gene=43%, lifetime risk of breast cancer=40%

Case 2

Sheila, aged 42, is concerned about her family history of breast cancer. She has read in a magazine about breast cancer genes and wants to be referred to a geneticist. Her mother, Shirley, had breast cancer diagnosed at age 62, three years ago. She had a lumpectomy and radiotherapy and is currently well. Her grandmother, May, had breast cancer diagnosed at age 73. She had a mastectomy but died two years later. One of May's sisters also had breast cancer diagnosed, at age 50, and is still alive (age 84); she has had no problems since her mastectomy. May's other sister is well, aged 88. Sheila's family view breast cancer as a family condition and have often discussed it in these terms. Now that she has read the magazine article she wonders if gene testing holds the answer. The article also talked about prophylactic mastectomy for people who are found to be gene carriers. Sheila has had no breast symptoms but worries about developing breast cancer. She is married with a son aged 15 and daughter aged 18. She also worries whether her daughter will get breast cancer. Sheila has a brother, who is well.

  Estimate of risk by Cyrillic program (version 2.1): heterozygote risk for BRCA gene=7%, lifetime risk of breast cancer=13%

  Lifetime risk of breast cancer for general population of women=9%20

We report a qualitative evaluation of RAGs (risk assessment in genetics), a computer program designed for primary care that draws pedigrees, assesses risk based on family history of breast and ovarian cancer, and suggests appropriate management. The software was developed through collaboration between JE and the ICRF Advanced Computation Laboratory using PROforma software. 14 15 The aims of this study were to explore general practitioners' opinions of using computers for genetic risk assessment and to study the use of RAGs in the consultation.

    Participants and methods
Top
Abstract
Introduction
Participants and methods
Results
Discussion
References

We chose an "action design" method, in which practitioners use prototype software in a work setting to inform necessary changes to the program.16 A purposive sample of 15 Oxfordshire general practitioners was recruited between May and September 1998 covering a range of computer literacy, interest in genetics, age, and sex.17 The doctors were identified by colleagues who work in the General Practice Research Group and invited to participate by JE by telephone. Only one doctor declined to participate in the study.

Semistructured interviews
JE performed the interviews, presenting himself as a researcher from the General Practice Research Group. Each doctor was first interviewed about his or her attitudes towards computers and involvement in genetics. The doctor was familiarised with RAGs and then used the program in two consultations with an actor playing a woman concerned about her family history of cancer (see box). The pedigrees were based on patients referred to the Oxford Cancer Genetics Clinic. The consultations were videotaped and followed by a semistructured interview with both doctor and actor, the video being used as a prompt to questioning.18 Interviews principally explored the ease of use of RAGs in the consultation and the effect of the program on the doctor-patient interaction. Issues identified during the earliest interviews defined the subjects explored in more depth in subsequent interviews, thus establishing the field validity and transferability of emerging concepts.19 Interviews were audiotaped and transcribed verbatim.

Analysis
The interview transcripts were entered into the NUD*IST program21 and analysed using a grounded theory approach22 by JE and RW with advice from SZ. During the analysis, JE and RW met regularly to discuss the emerging themes and resolve any differences in interpretation. Each transcript was open coded, whereby each phrase was analysed to create key categories. A coding framework was developed and modified during the analysis of subsequent interviews according to the emerging concepts. Relationships between different categories were identified by constant comparison between and within transcripts and by comparison with existing literature. Videotapes of the consultations were reviewed, and observations were integrated into the developing conceptual framework.23

All participants were sent a written summary of the analysis and invited to comment. We received responses from three doctors, who agreed with the main themes described. Additional minor comments helped refine the conceptual framework.

    Results
Top
Abstract
Introduction
Participants and methods
Results
Discussion
References

The table gives background information about the study participants.

Several themes emerged from the data: controlling the consultation, accessing the report from RAGs, managing the "third party effect," and concerns about time. These are presented in the context of two specific areas of inquiry: using computers for expert advice in genetics and using RAGs in the consultation. The box shows quotations selected from the interviews.


                              
View this table:
[in this window]
[in a new window]
 

Background information about the 15 general practitioners who participated in study. Values are medians (ranges) unless stated otherwise


Quotations from interviews

Using computers for expert advice in genetics

Doctor 3 (female, age 39): "I think the use of genetics could be a useful tool for us. I think you need to be guided by the genetic experts, those that are knowledgeable about it. We need to be able to get the right information and have the right tools at our disposal so we know what to do with that information and we know how to advise people. It's very easy to falsely reassure people without the information."

  Doctor 8 (male, age 36): "Because it can get very easily very complex, and I don't believe that we can really do that in our head, and assuming it gets updated regularly, [the computer advice] will be better than my knowledge I would assume, and it could be a source of help."

  Doctor 9 (female, age 44): "I probably hadn't put them [referral guidelines] in the right place, which is the fate of all bits of paper, so I tried, you know. I remembered one or two key points from it, but I couldn't remember the details so I had to phone up and ask."

RAGs in the consultation

Doctor 12 (female, age 29): "It was difficult because I was showing you the screen and it came up as I read, so I probably would have thought I'd quite liked to have read that and thought `How am I going to communicate this?' rather than showing you."

  Doctor 3: "Suddenly you get `High risk' coming up on the screen .... Its much nicer to say it in words without seeing `High risk' in black and white in front of you."

  Doctor 9: "Once I read the reasons why [it was low risk] I began to see that it was slightly awkward to get that information myself and deal with someone who was getting a bit agitated about this apparent conflict of advice."

  Doctor 3: "I was very aware she was sitting and I was saying nothing. I prefer to talk to people and look at them rather than spend a lot of time looking at the computer .... That made me feel uncomfortable."

  Doctor 5 (male, age 31): "If I know it's going to say `High risk' it might be [that] one of the ways round sort of breaking the news [would be that] you can invite the patient back to break news rather than suddenly there it is on the screen."

Using computers for expert advice in genetics
The participants viewed genetic information as specialist, complex knowledge pertaining to a new explanation of disease and allowing precise assessment of individuals' risk. The general practitioners accepted their traditional gatekeeper role in the new subject of genetics and felt it appropriate for them to identify people at increased genetic risk and to refer them for specialist counselling. They admitted that they found assessing genetic risk difficult and felt uncomfortable when doing so. This was true even of the two doctors who had attended courses on cancer genetics. The doctors managed this discomfort in a variety of ways: some referred all patients concerned about their family history of cancer, some assessed risk using a heuristic approach reflecting an incomplete memory of referral guidelines, and others attempted to reassure patients in the face of uncertainty.

The local cancer genetics guidelines, mailed to all Oxfordshire practices in 1997, were unfamiliar to most of the participants. Two doctors did recall them: one sat on the local medical council, which appraises all Oxfordshire guidelines, and the other was involved in a project evaluating an electronic version of local guidelines. Participants mentioned the problem of "guideline chaos," in which general practitioners were inundated with paper guidelines of varying quality, with resultant difficulty in storage and accessibility. Computers were seen as a potentially important method of implementing guidelines and restoring order to the perceived chaos. Furthermore, computers were seen to lend themselves to the complexity of genetic risk assessment for multifactorial diseases such as cancer.

Participants reflected on the skills required to use computers effectively in clinical practice and the limited training received. These included general keyboard and mouse skills, specific skills for particular programs, and, perhaps most importantly, the skill of using the computer within the consultation. Even the doctors who were more familiar with computers tended to enter little data during routine consultations because of pressures of time and concerns about the effect of the computer on doctor-patient communication (the "third party effect").

Using RAGs in the consultation
Most of the participants found RAGs easy to use, but certain errors were commonly made because of the interface. An important concept that emerged related to control of the consultation. The doctors felt relatively in control during data entry and tended to stop considering the meaning of the information they were entering into RAGs. Thus, when the report was generated, with the risk score highlighted amid other text, doctors were surprised and felt a sudden loss of control. When the high risk message for case 1 appeared on the screen, doctors commonly felt panic and wanted to turn the screen away from the "patient" to allow them to regain control and break the bad news more gently. With the low risk message of case 2, doctors were often surprised and, indeed, may have already tried to pre-empt the computer and warn the "patient" that she was at increased risk. This conflict between information given by the doctor and that provided by the computer further impaired the doctor's control of the consultation and made it difficult to reassure the low risk patient. The speed with which the computer generated the report added to this sense of losing control.

The doctors often felt unable to access additional information from the computer to support the management decision they were sharing with the patient. This related to poor highlighting and structure in the report. When they found the explanations provided by the program the doctors used this information to help regain control and reassure the patient.

Participants were concerned about the third party effect of the computer and missing important eye contact and subtle non-verbal cues. They were uncomfortable during silences when entering data and reading the report, which made them speak before they had a chance to assimilate the report fully. To counter the third party effect, doctors chose to share the computer screen with the patient. This worked well when generating the pedigree since there was a sense of joint effort and shared understanding. However, nearly all the doctors seemed uncomfortable sharing the screen when the report appeared, particularly for the high risk message.

Concerns about time were common, especially over recording a pedigree, as this was a period spent principally attending the "third party." The consultations took 10-25 minutes, depending on the doctors' computer skills and the actor's responses. The doctors proposed various time management strategies. Some accepted the natural variation in length of consultation in primary care, some suggested double appointments or staging the consultation, and others opted to delegate data entry (such as to the practice nurse). Staging and delegation had the advantage that they would avoid generating the risk report with the patient present. Thus, the doctors could maintain control of the consultation and could prepare themselves to break the bad news without sharing the information on screen with the patient for the first time.

    Discussion
Top
Abstract
Introduction
Participants and methods
Results
Discussion
References

This study explored the use of computer support for genetic advice in primary care, but several of its findings are relevant to the wider use of computers in general practice. Our sample of 15 doctors was purposively selected to represent a range of skills and opinions. Many of the themes that arose in this study about the role of primary care in genetics are common to those discovered in studies of general practitioners in Wessex and Nottingham (personal communication, S Kumar, E Watson). It is possible, therefore, that our findings are transferable to other general practitioners in the United Kingdom. However, qualitative research does not attempt to generalise but to provide a deeper understanding of phenomena and generate results of high validity.24 Although our participants used RAGs in specific scenarios, the use of simulated patients has been shown to predict clinical performance,25 while avoiding potential harm from using a prototype program with real patients. We used a combination of video-stimulated recall and rigorous data analysis using grounded theory. Such a method offers a simple, effective, and relatively cheap way of evaluating early prototypes and may be of value in the development of other general practice software.

Our participants thought RAGs was an appropriate application of information technology, given the complexity of genetic risk assessment for multifactorial disease and the detail of existing, paper based, referral guidelines. Using RAGs generated problems in the consultation because of the interface and the doctors' varying ability to integrate the computer into the consultation. The potential impact of computers on general practice consultations has been described, both in relation to sharing sensitive information on the computer screen26 and to doctor-patient communication.27 Difficulties arose when RAGs generated the report and the doctor experienced a sudden loss of control. This occurred because of concerns about risk being disclosed prematurely, because of uncertainty about how to discuss this information, and because of the inaccessibility of supporting information. Participants proposed a variety of strategies to avoid this situation by assimilating the report in the patient's absence.

RAGs is a generic tool for assessing genetic risk and will be extended to include other diseases such as colorectal cancer. This qualitative evaluation demonstrated the context in which RAGs will be used and the issues surrounding its integration into a general practice consultation. Many of the problems that arose were unpredictable and may not have been discovered using quantitative approaches. The development of medical information systems requires various stages of evaluation---investigating how the software functions and its impact on users, patients, and the health system.13 As a result of this study, we have identified important changes to the software to reduce the impact of the program on the consultation. We are now incorporating these changes and investigating the effect of RAGs on decision making.

    Acknowledgments

We thank the general practitioners who took part in this study and Mike Murphy and Joan Austoker for their advice and comments on the paper and design of the study.

Contributors: JE, RW, and SZ developed the study protocol. The decision process was designed and implemented by JE and DG using PROforma. AC wrote the family tree software. JF supported DG and AC in developing the software and use of PROforma. JE and RW analysed the data with guidance from SZ. All authors contributed to the final paper. JE is guarantor for the paper.

    Footnotes

Funding: The Cancer Research Campaign funded this study. JE is funded by the Cancer Research Campaign. RW, SZ, and JF are funded by the Imperial Cancer Research Fund. AC and DG are funded by the Economic and Social Research Council and the Imperial Cancer Research Fund.

Competing interest: None declared.

    References
Top
Abstract
Introduction
Participants and methods
Results
Discussion
References

1. Kinmonth AL, Reinhard J, Bobrow M, Pauker S. The new genetics: implications for clinical services in Britain and the United States. BMJ 1998; 316: 767-770[Free Full Text].
2. Harris R, Harris H. Primary care for patients at genetic risk. BMJ 1995; 311: 275-276[Free Full Text].
3. Fry A, Campbell H, Gudmundsdottir H, Rush R, Porteous M, Gorman D, et al. General practitioners' views on their role in cancer genetics and current practice. Fam Pract (in press).
4. Harris R, Harris H. Genetics in primary care. Report on workshop of EC Concerted Action on Genetic Services in Europe (CAGSE) in association with the RCGP Spring meeting, Blackpool, UK, 28 April 1995. J Med Genet 1996; 33: 346-348[Free Full Text].
5. Occasional paper 77. Genetics in primary care. North West England faculty of the RCGP. A report from the Faculty Genetics Group. London: Royal College of General Practitioners , 1998.
6. Emery J. Computer support for genetic advice in primary care. Br J Gen Pract (in press).
7. Fischer C, Schweigert S, Spreckelsen C, Vogel F. Programs, databases, and expert systems for human geneticists---a survey. Hum Genet 1996; 97: 129-137[Medline].
8. Johnston M, Langton K, Haynes B, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. Ann Int Med 1994; 120: 135-142[Abstract/Free Full Text].
9. Walton RT, Gierl C, Yudkin P, Mistry H, Vessey MP, Fox J. Evaluation of computer support for prescribing (CAPSULE) using simulated cases. BMJ 1997; 315: 791-795[Abstract/Free Full Text].
10. Sullivan F, Mitchell E. Has general practitioner computing made a difference to patient care? A systematic review of published reports. BMJ 1995; 311: 848-852[Abstract/Free Full Text].
11. Wyatt J, Spiegelhalter D. Evaluating medical expert systems: what to test and how? Med Inf 1990; 15: 205-217.
12. Heathfield H, Pitty D, Hanka R. Evaluating information technology in health care: barriers and challenges. BMJ 1998; 316: 1959-1961[Free Full Text].
13. Friedman CP, Wyatt J. Evaluation methods in medical informatics. New York: Springer , 1997.
14. Fox J, Johns N, Rahmanzadeh A. Disseminating medical knowledge: the PROforma approach. Artif Intell Med 1998; 14: 157-181[Medline].
15. Fox J, Johns N, Lyons C, Rahmanzadeh A, Thomson R, Wilson P. PROforma: a general technology for clinical decision support systems. Comput Methods Programs Biomed 1997; 54: 59-67[Medline].
16. Timpka T, Rauch E, Nyce JM. Towards productive knowledge-based systems in clinical organizations: a methods perspective. Artif Intell Med 1994; 6: 501-519[Medline].
17. Patton MQ. Qualitative evaluation and research methods. 2nd ed. Newbury Park, CA: Sage Publications , 1990.
18. Arborelius E, Timpka T. Comparison of patients' and doctors' comments on video-recorded consultations [see comments]. Scand J Primary Health Care 1991; 9: 71-77[Medline].
19. Mays N, Pope C. Qualitative research in health care. London: BMJ Publishing Group , 1996.
20. Cancer Research Campaign. Cancer incidence factsheet 1, 1998. London: CRC , 1998.
21. QSR NUD*IST [program]. version 4. Newbury Park, CA: Sage Publications , 1997.
22. Strauss AL, Corbin JM. Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage Publications , 1990.
23. Erikson F. Audiovisual records as a primary data source. Sociol Methods Res 1982; 11: 213-232. [Abstract]
24. Silverman D. Interpreting qualitative data: methods for analysing talk, text, and interaction. Thousand Oaks, CA: Sage Publications , 1993.
25. O'Hagan JJ, Davies LJ, Pears RK. The use of simulated patients in the assessment of actual clinical performance in general practice. N Z Med J 1986; 99: 948-951[Medline].
26. Ridsdale L, Hudd S. What do patients want and not want to see about themselves on the computer screen: a qualitative study. Scand J Primary Health Care 1997; 15: 180-183[Medline].
27. Greatbatch D, Heath C, Campion P, Luff P. How do desk-top computers affect the doctor-patient interaction? Fam Pract 1995; 12: 32-36[Abstract/Free Full Text].

(Accepted 26 March 1999)

© BMJ 1999
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to StumbleUpon StumbleUpon   Add to Technorati Technorati    What's this?

Relevant Articles

Designing and evaluating complex interventions to improve health care
Neil C Campbell, Elizabeth Murray, Janet Darbyshire, Jon Emery, Andrew Farmer, Frances Griffiths, Bruce Guthrie, Helen Lester, Phil Wilson, and Ann Louise Kinmonth
BMJ 2007 334: 455-459. [Extract] [Full Text] [PDF]

How computers can help to share understanding with patients
Frank Sullivan and Jeremy C Wyatt
BMJ 2005 331: 892-894. [Extract] [Full Text] [PDF]

Computer program may help in assessing genetic risk of cancer in primary care
BMJ 1999 319: 0. [Full Text]

This article has been cited by other articles:

  • Fox, J., Patkar, V., Chronakis, I., Begent, R. (2009). From practice guidelines to clinical decision support: closing the loop. JRSM 102: 464-473 [Full text]  
  • Hall, R., Saukko, P. M, Evans, P. H, Qureshi, N., Humphries, S. E (2007). Assessing family history of heart disease in primary care consultations: a qualitative study. Fam Pract 24: 435-442 [Abstract] [Full text]  
  • Campbell, N. C, Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., Kinmonth, A. L. (2007). Designing and evaluating complex interventions to improve health care. BMJ 334: 455-459 [Full text]  
  • Gramling, R. E., Vidrine, J. I. (2007). Risk Communication During Screening for Genomic Breast Cancer Susceptibility. AMERICAN JOURNAL OF LIFESTYLE MEDICINE 1: 54-58 [Abstract]  
  • Acheson, L. S., Zyzanski, S. J., Stange, K. C., Deptowicz, A., Wiesner, G. L. (2006). Validation of a Self-Administered, Computerized Tool for Collecting and Displaying the Family History of Cancer. JCO 24: 5395-5402 [Abstract] [Full text]  
  • Sullivan, F., Wyatt, J. C (2005). How computers can help to share understanding with patients. BMJ 331: 892-894 [Full text]  
  • U.S. Preventive Services Task Force*, (2005). Genetic Risk Assessment and BRCA Mutation Testing for Breast and Ovarian Cancer Susceptibility: Recommendation Statement. ANN INTERN MED 143: 355-361 [Abstract] [Full text]  
  • Nelson, H. D., Huffman, L. H., Fu, R., Harris, E. L. (2005). Genetic Risk Assessment and BRCA Mutation Testing for Breast and Ovarian Cancer Susceptibility: Systematic Evidence Review for the U.S. Preventive Services Task Force. ANN INTERN MED 143: 362-379 [Abstract] [Full text]  
  • Walter, F. M., Emery, J., Braithwaite, D., Marteau, T. M. (2004). Lay Understanding of Familial Risk of Common Chronic Diseases: A Systematic Review and Synthesis of Qualitative Research. Ann Fam Med 2: 583-594 [Abstract] [Full text]  
  • Walter, F. M, Britten, N. (2002). Patients' understanding of risk: a qualitative study of decision-making about the menopause and hormone replacement therapy in general practice. Fam Pract 19: 579-586 [Abstract] [Full text]  
  • Braithwaite, D., Sutton, S., Smithson, W H., Emery, J. (2002). Internet-based risk assessment and decision support for the management of familial cancer in primary care: a survey of GPs' attitudes and intentions. Fam Pract 19: 587-590 [Abstract] [Full text]  
  • Kinzie, M. B., Cohn, W. F., Julian, M. F., Knaus, W. A. (2002). A User-centered Model for Web Site Design: Needs Assessment, User Interface Design, and Rapid Prototyping. J. Am. Med. Inform. Assoc. 9: 320-330 [Abstract] [Full text]  
  • Pagon, R. A, Pinsky, L., Beahler, C. C (2001). Online medical genetics resources: a US perspective. BMJ 322: 1035-1037 [Full text]  
  • Women's Concerns Study Group, (2001). Raising concerns about family history of breast cancer in primary care consultations: prospective, population based study. BMJ 322: 27-28 [Full text]  
  • Shere, M H, Gray, E., Rothnie, N., Fowler, A. (2000). Family histories of cancer in primary care. BMJ 321: 955-955 [Full text]  
  • Bury, J., Fox, J. (2000). Towards a quality and safety framework for point of care decision support systems. Health Informatics Journal 6: 134-138 [Abstract]  
  • Emery, J., Walton, R., Murphy, M., Austoker, J., Yudkin, P., Chapman, C., Coulson, A., Glasspool, D., Fox, J. (2000). Computer support for interpreting family histories of breast and ovarian cancer in primary care: comparative study with simulated cases. BMJ 321: 28-32 [Abstract] [Full text]  

Rapid Responses:

Read all Rapid Responses

Control of interaction after risk assessment
Virginia R Hetrick
bmj.com, 4 Jul 1999 [Full text]
Control of interaction after risk assessment
Virginia R Hetrick
bmj.com, 4 Jul 1999 [Full text]
Access jobs at BMJ Careers
Whats new online at Student BMJ