BMJ 2002;325:679 ( 28 September )

Papers

Using standardised patients to measure physicians' practice: validation study using audio recordings

Jeff Luck, assistant professor, ^a John W Peabody, deputy director. ^b

^a Veterans Administration, Greater Los Angeles Healthcare System, 11 301 Wilshire Blvd, Los Angeles, CA 90073, USA, ^b Institute for Global Health, 74 New Montgomery St, San Francisco, CA 94105, USA

Correspondence to: John W Peabody
peabody{at}psg.ucsf.edu

Objective: To assess the validity of standardised patients to measure the quality of physicians' practice.
Design: Validation study of standardised patients' assessments. Physicians saw unannounced standardised patients presenting with common outpatient conditions. The standardised patients covertly tape recorded their visit and completed a checklist of quality criteria immediately afterwards. Their assessments were compared against independent assessments of the recordings by a trained medical records abstractor.
Setting: Four general internal medicine primary care clinics in California.
Participants: 144 randomly selected consenting physicians.
Main outcome measures: Rates of agreement between the patients' assessments and independent assessment.
Results: 40 visits, one per standardised patient, were recorded. The overall rate of agreement between the standardised patients' checklists and the independent assessment of the audio transcripts was 91% (=0.81). Disaggregating the data by medical condition, site, level of physicians' training, and domain (stage of the consultation) gave similar rates of agreement. Sensitivity of the standardised patients' assessments was 95%, and specificity was 85%. The area under the receiver operator characteristic curve was 90%.
Conclusions: Standardised patients' assessments seem to be a valid measure of the quality of physicians' care for a variety of common medical conditions in actual outpatient settings. Properly trained standardised patients compare well with independent assessment of recordings of the consultations and may justify their use as a "gold standard" in comparing the quality of care across sites or evaluating data obtained from other sources, such as medical records and clinical vignettes.

What is already known on this topic Standardised patients are valid and reliable reporters of physicians' practice in the medical education setting However, validating standardised patients' measurements of quality of care in actual primary practice is more difficult and has not been done in a prospective study What this study adds Reports of physicians' quality of care by unannounced standardised patients compare well with independent assessment of the consultations

---

© BMJ 2002
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

Standardised patients are a valid measure of quality of care
BMJ 2002 325: 0. [Full Text] [PDF]

Communications and emotions

Robert Buckman
BMJ 2002 325: 672. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Kelly, F. S, Williams, K. A, Benrimoj, S. I (2009). Does Advice from Pharmacy Staff Vary According to the Nonprescription Medicine Requested?. The Annals of Pharmacotherapy 43: 1877-1886 [Abstract] [Full text]  
• Weaver, M R, Myaya, M, Disasi, K, Regoeng, M, Matumo, H N, Madisa, M, Puttkammer, N, Speilberg, F, Kilmarx, P H, Marrazzo, J M (2008). Routine HIV testing in the context of syndromic management of sexually transmitted infections: outcomes of the first phase of a training programme in Botswana. Sex. Transm. Infect. 84: 259-264 [Abstract] [Full text]  
• Leon, F. R., Lundgren, R., Jennings, V. (2008). Provider Selection of Evidence-Based Contraception Guidelines in Service Provision: A Study in India, Peru, and Rwanda. Eval Health Prof 31: 3-21 [Abstract]  
• Hepner, K. A., Rowe, M., Rost, K., Hickey, S. C., Sherbourne, C. D., Ford, D. E., Meredith, L. S., Rubenstein, L. V. (2007). The Effect of Adherence to Practice Guidelines on Depression Outcomes. ANN INTERN MED 147: 320-329 [Abstract] [Full text]  
• Leon, F. R., Arevalo, M., Lundgren, R., Jennings, V., Huapaya, A., Panfichi, R. (2007). Four Criteria to Evaluate Providers' Service-Delivery Response to New Contraceptive Introduction. Eval Rev 31: 364-390 [Abstract]  
• Giesen, P., Ferwerda, R., Tijssen, R., Mokkink, H., Drijver, R., van den Bosch, W., Grol, R. (2007). Safety of telephone triage in general practitioner cooperatives: do triage nurses correctly estimate urgency?. Qual Saf Health Care 16: 181-184 [Abstract] [Full text]  
• Eagles, J. M., Calder, S. A., Wilson, S., Murdoch, J. M., Sclare, P. D. (2007). Simulated patients in undergraduate education in psychiatry. Psychiatr. Bull. 31: 187-190 [Full text]  
• Klass, D. (2007). Assessing Doctors at Work -- Progress and Challenges. NEJM 356: 414-415 [Full text]  
• Jorg, F., Borgers, N., Schrijvers, A. J. P., Hox, J. J. (2006). Variation in Long-Term Care Needs Assessors' Willingness to Support Clients' Requests for Admission to a Residential Home: A Vignette Study. J Aging Health 18: 767-790 [Abstract]  
• Mularski, R. A., Asch, S. M., Shrank, W. H., Kerr, E. A., Setodji, C. M., Adams, J. L., Keesey, J., McGlynn, E. A. (2006). The Quality of Obstructive Lung Disease Care for Adults in the United States as Measured by Adherence to Recommended Processes. Chest 130: 1844-1850 [Abstract] [Full text]  
• Asch, S. M., Kerr, E. A., Keesey, J., Adams, J. L., Setodji, C. M., Malik, S., McGlynn, E. A. (2006). Who is at greatest risk for receiving poor-quality health care?. NEJM 354: 1147-1156 [Abstract] [Full text]  
• Berner, E. S., Houston, T. K., Ray, M. N., Allison, J. J., Heudebert, G. R., Chatham, W. W., Kennedy, J. I. Jr., Glandon, G. L., Norton, P. A., Crawford, M. A., Maisiak, R. S. (2006). Improving Ambulatory Prescribing Safety with a Handheld Decision Support System: A Randomized Controlled Trial. J. Am. Med. Inform. Assoc. 13: 171-179 [Abstract] [Full text]  
• Leon, F. R., Rios, A., Zumaran, A. (2005). Training x Trainee Interactions in a Family Planning Intervention. Eval Rev 29: 576-590 [Abstract]  
• Franks, P., Fiscella, K., Shields, C. G., Meldrum, S. C., Duberstein, P., Jerant, A. F., Tancredi, D. J., Epstein, R. M. (2005). Are Patients' Ratings of Their Physicians Related to Health Outcomes?. Ann Fam Med 3: 229-234 [Abstract] [Full text]  
• Kravitz, R. L., Epstein, R. M., Feldman, M. D., Franz, C. E., Azari, R., Wilkes, M. S., Hinton, L., Franks, P. (2005). Influence of Patients' Requests for Direct-to-Consumer Advertised Antidepressants: A Randomized Controlled Trial. JAMA 293: 1995-2002 [Abstract] [Full text]  
• Asch, S. M., McGlynn, E. A., Hogan, M. M., Hayward, R. A., Shekelle, P., Rubenstein, L., Keesey, J., Adams, J., Kerr, E. A. (2004). Comparison of Quality of Care for Patients in the Veterans Health Administration and Patients in a National Sample. ANN INTERN MED 141: 938-945 [Abstract] [Full text]  
• Peabody, J. W., Luck, J., Glassman, P., Jain, S., Hansen, J., Spell, M., Lee, M. (2004). Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study. ANN INTERN MED 141: 771-780 [Abstract] [Full text]  
• Kerr, E. A., McGlynn, E. A., Adams, J., Keesey, J., Asch, S. M. (2004). Profiling The Quality Of Care In Twelve Communities: Results From The CQI Study. Health Aff (Millwood) 23: 247-256 [Abstract] [Full text]  
• McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., Kerr, E. A. (2003). The Quality of Health Care Delivered to Adults in the United States. NEJM 348: 2635-2645 [Abstract] [Full text]  
• Buckman, R. (2002). Communications and emotions. BMJ 325: 672-672 [Full text]  

Online poll

Find out more on doc2doc >>