Assessing clinical competence and revalidation of clinicians

BMJ 2001; 322 doi: https://doi.org/10.1136/bmj.322.7302.1600 (Published 30 June 2001) Cite this as: BMJ 2001;322:1600

Simulated surgeries would have advantages

  1. Peter Burrows, general practitioner (pjburrows@interalpha.co.uk)
  1. Abbey Mead Surgery, Romsey, Hampshire SO51 8EN
  2. Measham Medical Unit, Measham DE12 7HR
  3. Department of General Practice and Primary Health Care, Leicester Warwick Medical School, University of Leicester, Leicester General Hospital, Leicester LE5 4PW
  4. Office of the Medical Director, Guys and St Thomas NHS Trust, Guy's Hospital, London SE1 9RT

    EDITOR—McKinley et al present cogent arguments for directly assessing the consultation competence of general practitioners undergoing revalidation.1 They identify the required attributes of an assessment as reliability, validity, acceptability, feasibility, and educational impact. They do not mention, however, that the methodology must also be capable of setting a consistent standard for pass/fail decisions for every candidate.

    The model that they offer depends on the judgment of two observers watching 10 consecutive consultations, a task that they might undertake 12 times in a year. If the prevalence of insufficient competence was 2% they might encounter it once every four years. How could they apply a minimum standard reliably, along with their 1000 assessor colleagues? Just 10 randomly presenting cases will be too few to guarantee an adequate coverage of consultation skills, and in the absence of standardisation it will be hard to achieve a reliability score above 0.5. Furthermore, the patients will behave differently if they know that their doctor is under assessment, thus compromising validity.

    The 12 case simulated surgery for the MRCGP (membership of the Royal College of General Practitioners) examination is a model that would overcome these difficulties. The cases are written to test specific skills; the patients are role players who are trained to present the same challenge to each candidate; each examiner observes the same case throughout the circuit and marks the candidate's performance on a predetermined schedule; each candidate is assessed by all 12 examiners, which minimises bias; and the pass mark is set by the aggregation of the judgments of all the examiners, based on the cases they have marked.

    This examination has yielded reliability figures (coefficient α) in excess of 0.8 in all six administrations over three years. Face validity is high (“like a locum …

    View Full Text

    Sign in

    Log in through your institution