Use of SPRAT for peer review of paediatricians in training

doi:10.1136/bmj.38447.610451.8F

BMJ 2005;330:1251-1253 (28 May), doi:10.1136/bmj.38447.610451.8F (published 9 May 2005)

Learning in practice

Use of SPRAT for peer review of paediatricians in training

Julian C Archer, clinical research fellow¹, John Norcini, president³, Helena A Davies, consultant in medical education²

¹ Academic Unit of Child Health, Sheffield Children's Hospital, Sheffield S10 2HT, ² Postgraduate Medical Education Department, Sheffield Children's Hospital, ³ Foundation for the Advancement of International Medical Education Research, (FAIMER), 3624 Market Street, Philadelphia, PA 19104, USA

Correspondence to: H A Davies h.davies{at}sheffield.ac.uk

Abstract

Objective To determine whether a multisource feedback questionnaire,SPRAT (Sheffield peer review assessment tool), is a feasibleand reliable assessment method to inform the record of in-trainingassessment for paediatric senior house officers and specialistregistrars.

Design Trainees' clinical performance was evaluated using SPRATsent to clinical colleagues of their choosing. Responses wereanalysed to determine variables that affected ratings and theirmeasurement characteristics.

Setting Three tertiary hospitals and five secondary hospitalsacross a UK deanery.

Participants 112 paediatric senior house officers and middlegrades.

Main outcome measures 95% confidence intervals for mean ratings;linear and hierarchical regression to explore potential biasingfactors; time needed for the process per doctor.

Results 20 middle grades and 92 senior house officers were assessedusing SPRAT to inform their record of in-training assessment;921/1120 (82%) of their proposed raters completed a SPRAT form.As a group, specialist registrars (mean 5.22, SD 0.34) scoredsignificantly higher (t = – 4.765) than did senior houseofficers (mean 4.81, SD 0.35) (P < 0.001). The grade of thedoctor accounted for 7.6% of the variation in the mean ratings.The hierarchical regression showed that only 3.4% of the variationin the means could be additionally attributed to three mainfactors (occupation of rater, length of working relationship,and environment in which the relationship took place) when thedoctor's grade was controlled for (significant F change <0.001). 93 (83%) of the doctors in this study would have neededonly four raters to achieve a reliable score if the intent wasto determine if they were satisfactory. The mean time takento complete the questionnaire by a rater was six minutes. Justover an hour of administrative time is needed for each doctor.

Conclusions SPRAT seems to be a valid way of assessing largenumbers of doctors to support quality assurance procedures fortraining programmes. The feedback from SPRAT can also be usedto inform personal development planning and focus quality improvements.

Add to CiteULike CiteULike Add to Complore Complore Add to Connotea Connotea Add to Del.icio.us Del.icio.us Add to Digg Digg Add to Reddit Reddit Add to Technorati Technorati What's this?

Relevant Articles

Feedback tool assessed junior doctors reliably
BMJ 2005 330: 0. [Full Text]

Can poorly performing doctors blame their assessment tools?: Richard Baker
BMJ 2005 330: 1254. [Full Text] [PDF]

Review of instruments for peer assessment of physicians: Richard Evans, Glyn Elwyn, and Adrian Edwards
BMJ 2004 328: 1240. [Abstract] [Full Text] [PDF]

Multisource feedback: a method of assessing surgical practice: Claudio Violato, Jocelyn Lockyer, and Herta Fidler
BMJ 2003 326: 546-548. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

McGraw, M. E (2009). Delivery of the paediatric curriculum of the Royal College of Paediatrics and Child Health (RCPCH). Arch. Dis. Child. 94: 254-257 [Full text]
Campbell, J L, Richards, S H, Dickens, A, Greco, M, Narayanan, A, Brearley, S (2008). Assessing the professional performance of UK doctors: an evaluation of the utility of the General Medical Council patient and colleague questionnaires. Qual Saf Health Care 17: 187-193 [Abstract] [Full text]
Archer, J. (2008). mini-PAT (Peer Assessment Tool): a well kept secret?. JRSM 101: 272-272 [Full text]
Abdulla, A. (2008). A critical analysis of mini peer assessment tool (mini-PAT). JRSM 101: 22-26 [Abstract] [Full text]
Naeem, A., Kent, A., Vijayakrishnan, A. (2007). Foundation programme assessment tools in psychiatry. Psychiatr. Bull. 31: 427-430 [Full text]
Galbraith, R M, Holtman, M C, Clyman, S G (2006). Use of assessment to reinforce patient safety as a habit. Qual Saf Health Care 15: i30-i33 [Abstract] [Full text]
Violato, C., Lockyer, J. M., Fidler, H. (2006). Assessment of Pediatricians by a Regulatory Authority. Pediatrics 117: 796-802 [Abstract] [Full text]
Baker, R. (2005). Can poorly performing doctors blame their assessment tools?. BMJ 330: 1254-1254 [Full text]