BMJ 2003;326:319-321 ( 8 February )

Learning in practice

Validation of the Fresno test of competence in evidence based medicine

Kathleen D Ramos, assistant professor, a Sean Schafer, associate professor, a Susan M Tracz, professor. b

a University of California San Francisco, Fresno Medical Education Program, Department of Family and Community Medicine, 445 South Cedar Avenue, Fresno, CA 93702, USA, b California State University Fresno, Kremen School of Education and Human Development, 5005 North Maple Avenue, Fresno, CA 93740, USA

Correspondence to: K D Ramos katie.ramos{at}ucsfresno.edu

Objective: To describe the development and validation of a test of knowledge and skills in evidence based medicine.
Design: Cross sectional study.
Setting: Family practice residency programme in California; a list server for those who teach evidence based medicine; and an evidence based medicine seminar series.
Participants: Family practice residents and faculty members (n=43); volunteers self identified as experts in evidence based medicine (n=53); family practice teachers (19) beginning a seminar series on evidence based medicine.
Intervention: The Fresno test is a performance based measure for use in medical education that assesses a wide range of evidence based medicine skills. Open ended questions are scored with standardised grading rubrics. Calculation skills are assessed by fill in the blank questions.
Main outcome measures: Inter-rater reliability, internal reliability, item analyses, and construct validity.
Results: Inter-rater correlations ranged from 0.76 to 0.98 for individual items. Cronbach's alpha  was 0.88. Item difficulties ranged from moderate to difficult, all with positive and strong ability to discriminate between candidates. Experts scored consistently higher than novices. On the 212 point test, the novice mean was 95.6 and the expert mean was 147.5 (P<0.001). On individual items, a higher proportion of experts than novices earned passing scores on 15 of the 17 items.
Conclusion: The Fresno test is a reliable and valid test for detecting the effect of instruction in evidence based medicine. Its use in other settings requires further exploration.

What is already known on this topic
Instruction in evidence based medicine is provided in many medical education settings, but it effectiveness is unknown

Existing measures to assess competence tend to be narrowly focused and of uncertain validity

What this study adds
The Fresno test measures a wide range of knowledge and skills necessary for evidence based practice

The standardised grading systems produced a high degree of consistency between graders

Experts scored significantly higher on the test than novices in evidence based medicine, showing that the test has construct validity




© 2003 BMJ Publishing Group Ltd
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to StumbleUpon StumbleUpon   Add to Technorati Technorati   Add to Twitter Twitter    What's this?

Relevant Article

Competent in evidence based medicine? Take the Fresno test
BMJ 2003 326: 0. [Full Text]

This article has been cited by other articles:

  • Adeodu, A., Agius, R., Madan, I. (2009). Attitudes and barriers to evidence-based guidelines among UK occupational physicians. Occup Med (Lond) 59: 586-592 [Abstract] [Full text]  
  • Cimpean, D., Bartels, S. J., Merrens, M. R., Drake, R. E., Garrity, W. (2009). Summer Institute in Evidence-Based Psychiatry and Mental Health. Acad. Psychiatry 33: 327-330 [Full text]  
  • Coverdale, J. H., Weiss Roberts, L., Louie, A. K. (2008). Teaching Evidence-Based Psychiatry to Residents and Fellows: Developing the Curriculum. Acad. Psychiatry 32: 453-457 [Full text]  
  • Agrawal, S., Szatmari, P., Hanson, M. (2008). Teaching Evidence-Based Psychiatry: Integrating and Aligning the Formal and Hidden Curricula. Acad. Psychiatry 32: 470-474 [Abstract] [Full text]  
  • Mascola, A. J. (2008). Guided Mentorship in Evidence-Based Medicine for Psychiatry: A Pilot Cohort Study Supporting a Promising Method of Real-Time Clinical Instruction. Acad. Psychiatry 32: 475-483 [Abstract] [Full text]  
  • Feinstein, R. E., Rothberg, B., Weiner, N., Savin, D. M. (2008). University of Colorado Department of Psychiatry Evidence-Based Medicine Educational Project. Acad. Psychiatry 32: 525-530 [Full text]  
  • Rodrigo, G. J., Nannini, L. J., Rodriguez-Roisin, R. (2008). Safety of Long-Acting {beta}-Agonists in Stable COPD: A Systematic Review. Chest 133: 1079-1087 [Abstract] [Full text]  
  • Poolman, R. W., Sierevelt, I. N., Farrokhyar, F., Mazel, J. A., Blankevoort, L., Bhandari, M. (2007). Perceptions and Competence in Evidence-Based Medicine: Are Surgeons Getting Better? A Questionnaire Survey of Members of the Dutch Orthopaedic Association. JBJS 89: 206-215 [Abstract] [Full text]  
  • Shaneyfelt, T., Baum, K. D., Bell, D., Feldstein, D., Houston, T. K., Kaatz, S., Whelan, C., Green, M. (2006). Instruments for evaluating education in evidence-based practice: a systematic review.. JAMA 296: 1116-1127 [Abstract] [Full text]  
  • Chiappelli, F., Prolo, P., Rosenblum, M., Edgerton, M., Cajulis, O. S. (2006). Evidence-Based Research in Complementary and Alternative Medicine II: The Process of Evidence-Based Research.. Evid Based Complement Alternat Med 3: 3-12 [Abstract] [Full text]  
  • Green, M. L. (2005). A Train-the-Trainer Model for Integrating Evidence-Based Medicine Training into Podiatric Medical Education. J. Am. Podiatr. Med. Assoc. 95: 497-504 [Abstract] [Full text]  
  • Louie, A., Weiss Roberts, L., Coverdale, J. (2005). Imparting the Knowledge, Science, and Art of Psychopharmacology: Thoughts on Educational Research Questions. Acad. Psychiatry 29: 113-115 [Full text]  
  • Coverdale, J., Weiss Roberts, L., Louie, A. (2004). Committing to the Scientific Basis of Psychiatry: Implications for Training and Educational Research. Acad. Psychiatry 28: 261-262 [Full text]  
  • Straus, S. E, Green, M. L, Bell, D. S, Badgett, R., Davis, D., Gerrity, M., Ortiz, E., Shaneyfelt, T. M, Whelan, C., Mangrulkar, R., the Society of General Internal Medicine Evidence-, (2004). Evaluating the teaching of evidence based medicine: conceptual framework. BMJ 329: 1029-1032 [Full text]  
  • Liu, J. Y., Woloshin, S., Laycock, W. S., Rothstein, R. I., Finlayson, S. R. G., Schwartz, L. M. (2004). Symptoms and Treatment Burden of Gastroesophageal Reflux Disease: Validating the GERD Assessment Scales. Arch Intern Med 164: 2058-2064 [Abstract] [Full text]  

Rapid Responses:

Read all Rapid Responses

Does Fresno assess real-life EBM skills?
Caroline Mawer
bmj.com, 17 Feb 2003 [Full text]
The Fresno test
Robert K McKinley, et al.
bmj.com, 17 Feb 2003 [Full text]
Access jobs at BMJ Careers
Whats new online at Student BMJ