Recertification in the United States

Patient outcomes

The assessment of patient outcomes is the most important component of a recertification programme. It directly reassures the public that doctors are performing well, and it is tailored to practice so it offers evaluation of what doctors actually do, rather than what they do in an artificial testing situation.

In the United States, outcomes assessment has become a reality of practice Many healthcare systems give doctors a “report card” detailing their performance in areas such as screening, prescribing, and patient satisfaction.6^–8 However, outcomes assessment for a national recertification programme faces significant technical obstacles in data collection and in the number of cases that need to be sampled to have confidence in the results.9 Moreover, there are difficulties in evaluating the outcomes themselves, including attribution, complexity, and case mix.10 Treatment is often provided by healthcare teams, so it is difficult to attribute a particular patient outcome to a single doctor. In addition, patients with the same condition often vary in complexity for a variety of reasons including the severity of the disease, comorbid conditions, and patient compliance. Furthermore, there is considerable variation in the patient mix from one doctor to another Although there are partial solutions to these problems, a rigorous and fair evaluation based on patient outcomes is not yet possible.10

This inability to do adequate assessment in a setting where so much is at stake for patients and doctors has created a conundrum for the boards. In response they have introduced methods that incorporate assessment of practice performance into their recertification programmes but which are not quite as rigorous as typical examination methods. Also, they have focused on aspects of the process of care that are known to correlate with patient outcomes rather than on the outcomes themselves.1112 For example, the American Board of Ophthalmology (www.abop.org) has an outcomes related process measure as part of its recertification programme The purpose of this measure is to assess whether office records conform to standards of practice. For specified conditions (for example, macular degeneration, retinal detachment) ophthalmologists audit the records of five patients and indicate on a scan sheet, tailored to that condition, various relevant aspects of the patient's history, examination, diagnostic procedures, assessment, and management. The forms are sent to the board, where they are scored, and feedback is given to candidates. Candidates must also send photocopies of the initial and most recent visits, with the identification of the patients masked, and the board reserves the right to visit the candidate's office to verify the validity of the material.

The American Board of Internal Medicine (www.abim.org) plans to use a similar methodology for common conditions (for example, diabetes, hypertension) and procedures (for example, flexible sigmoidoscopy) It already has available a clinical preventive services module as a self selected part of its recertification programme. The core of the module is also an audit (conducted by the candidate) of the candidate's patient records for selected preventive services. After completing the audit the candidate must devise a quality improvement plan. The board reserves the right to ask for copies of the patient records underlying the audit.

Finally, as part of its recertification programme the American Board of Emergency Medicine had, up to 1993, an oral examination based on a doctor's patient records—called chart stimulated recall.13 Doctors in accident and emergency submitted a series of charts from their practice. The board selected several and these formed the basic test material for the oral examination. Candidates were questioned about their actions and omissions as well as their reasoning in coming to diagnostic and therapeutic decisions.

Medical knowledge and judgment

Ideal measures of outcomes would ensure that doctors are capably treating patients with common conditions. By themselves, however, they are inadequate for recertification. Firstly, medical knowledge is constantly evolving and it would take years, at considerable cost to patients, to detect doctors who are not keeping up. Secondly, a sizeable portion of most doctors' practices is devoted to the diagnosis and treatment of infrequent but clinically important conditions. This work will not be captured in an assessment of practice outcomes because there will be too few instances to have confidence in the meaning of the results. Finally, certification indicates competence in wide ranging disciplines, such as internal medicine, but doctors naturally tend to narrow their practices over time. An assessment of outcomes does not assure competence in the broader domain within which the specialist is entitled to practice.

In response to these issues, a recertification programme should include a secure monitored test of competence. At the moment most boards in the United States have some form of written examination. A few boards offer oral examination as an alternative, but candidates are reluctant to submit themselves to this type of assessment; they prefer the written examination. The trend in the United States is to offer these examinations on computer. For example, the American Board of Anesthesiology (www.abanes.org) currently offers its recertification examination on computers at sites around the country for a restricted period of time. Several of the boards are now planning to do likewise and to have the examination available for several periods of time throughout the year. This will ease access, offer quicker feedback, allow the inclusion of video based and audio based test material, and permit rapid retakes of the examinations for those who are unsuccessful.

Professionalism

Assessments of outcomes, medical knowledge, and judgment address themselves mainly to doctors' technical skills. Of equal importance are ethical conduct and the quality of the relationship between doctor and patient. An assessment of these aspects of competence is important because patients are vulnerable to inefficient doctors or those whose conduct is unethical. These are unlikely to be captured in outcomes So too is any evaluation of how well the doctor communicates with patients and whether he or she treats them with compassion and respect. Therefore it is essential that a recertification programme includes an evaluation of professionalism.

Most boards address this need by requiring a valid licence to practise medicine and ensuring that no disciplinary actions have been taken against the candidate.14 Additionally, testimonials from local credentialing bodies, such as hospitals, are often collected.

As a self selected part of its recertification programme, the American Board of Internal Medicine has a patient satisfaction and peer assessment module. Based on work funded by the board, candidates are sent rating forms for their peers and patients.15^–17 The peer associate rating form collects information related to technical and non-technical skills, particularly professionalism. The patient satisfaction questionnaire concentrates on integrity, compassion, and respect. Both forms comprise 10 to 15 items and a few demographic questions.

Previous research has shown that selection of peer raters by the doctor does not bias the final results, so candidates distribute the forms themselves.16 Respondents call a free telephone number and interactive voice response technology collects their answers to each of the questions on the form. When they have ratings from 10 peers and 25 patients, the candidates have completed the module. Feedback to candidates is currently a summary of the ratings. Further information about completing the patient satisfaction and peer assessment module appears on the BMJ's website. Plans for the future include normative information (how a doctor compares with his or her peers) and ultimately the development of standards for performance. Preliminary work has been successful and the board has voted to make this part of recertification mandatory over the next five years.