Learning in practice

Competency based medical training: review

Commentary: The baby is thrown out with the bathwater

Competency based medical training: review

Wai-Ching Leung, honorary lecturer in public health medicine. 

Medicine, Health Policy and Practice, University of East Anglia, Norwich NR4 7TJ

wai_chingleung{at}hotmail.com

The competency approach has become prominent at most stages of undergraduate and postgraduate medical training in many countries. In the United Kingdom, for example, it forms part of the performance procedures of the General Medical Council (GMC),¹ underpins objectively structured clinical examinations (OSCEs) and records of in-training assessment (RITA), and has been advocated for the selection of registrars in general practice and interviews. ^{2 3} It has become central to the professional lives of all doctors and is treated as if it were a panaceabut there is little consensus among trainees, trainers, and committees on what this approach entails.

I aim to explore the origins and development of the competency approach, evaluate its current role in medical training, and discuss its strengths and limitations.

Summary points The competency based approach consists of functional analysis of occupational roles, translation of these roles into outcomes, and assessment of trainees' progress on the basis of their demonstrated performance of these outcomes It has become dominant at most stages of medical training Potential advantages include individualised flexible training, transparent standards, and increased public accountability If applied inappropriately, it can result in demotivation, a focus on minimum acceptable standards, increased administrative burden and a reduction in the educational content We should be cautious of applying the competency based approach universally unless robustly defined higher order competencies are available

	The birth of the competency movement

Top The birth of the... The rise of "holistic"... Current scene in medical... An evaluation References

The competency approach did not result directly from recent scandals of incompetent doctors. It originated from parallel developments in vocational training in many countries, such as the national qualifications framework in New Zealand, the national training board in Australia, the national skills standards initiative in the United States, and the national vocational qualifications (NVQs) in the United Kingdom.⁴ This movement was driven largely by the political perceived need to make the national workforce more competitive in the global economy. For example, in Britain, the national vocational qualifications were developed as a set of standards each broken down into elements by which performance in the workplace can be assessed. This approach has since been adopted for training across other areas, particularly the technical and vocational fields.

How does competency based training work? The basic essential elements consist of functional analysis of the occupational roles, translation of these roles ("competencies") into outcomes, and assessment of trainees' progress in these outcomes on the basis of demonstrated performance. Progress is defined solely by the competencies achieved and not the underlying processes or time served in formal educational settings.⁵ Assessments are based on a set of clearly defined outcomes so that all parties concerned, including assessors and trainees, can make reasonably objective judgments about whether or not each trainee has achieved them.⁶ Potential benefits of this approach include individualised flexible training and transparent standards.

This approach has attracted several criticisms. Firstly, functional analysis of occupational roles is problematic. It is difficult to identify a range of competencies that truly cover work roles in their broadest sense and to represent adequately the types of knowledge relevant to the competency identified. ^{7 8} Secondly, the assessment of competencies is by no means value free, and people who use it shape its meaning. Thirdly, the competency approach is based primarily on the behaviourist framework, which attempts to break down work roles into small discrete tasks. It ignores the connections between individual tasks and the meaning underlying each task. It therefore cannot represent the complex nature of situations in the real world. The danger is that these narrowly defined competencies will dominate the curriculum, which would not be suitable for learning in higher education.⁹ The approach using checklists and passing or failing candidates is superficial and often proves demotivating, as it encourages trainees to do the right thing to pass rather than to think critically and excel. The parties concernedtrainees, employers, professional bodies, and the governmentmay have different views about which aspect of the occupation is regarded as the most important.¹⁰ The process of developing competencies is at least partly political because it allows the government to influence what are included as important competencies and to allocate resources based on outcomes of performance. ^{11 12} A recent review of published evaluative studies of competency based training found an increase in administrative burden but no convincing beneficial effects on motivating students, work performance, or relevance to the needs of industry.¹¹

	The rise of "holistic" varieties

Top The birth of the... The rise of "holistic"... Current scene in medical... An evaluation References

As this behaviourist approach to learning would be even less appropriate for professions requiring complex skills, a range of broader competency approaches flourished. In 1991, the general national vocational qualifications, which include core skills such as numeracy, communication, and problem solving, were developed to supplement the NVQ framework, although doubts exist about whether such generic skills transferable to all context actually exist.⁹ An integrated approach acknowledges competency as a complex combination of knowledge, attitudes, skills, and personal values.¹³ A holistic approach takes into account the cultural and social context in assessing competence and focuses on how personal attributes are used to achieve outcomes in real life scenarios.¹⁴ A competency of a higher ordermeta-competencyhas been used to describe the general ability to learn and apply competencies effectively in many different aspects of a person's activities.¹⁵ These approaches attempt to make the competency based model less reductionist in nature.

	Current scene in medical training

Top The birth of the... The rise of "holistic"... Current scene in medical... An evaluation References

Traditionally, the framework of medical training was time based, and students were assessed periodically to determine their grades. Equal weight was given to both process and outcome of learning. Emphasis was given to the understanding of basic concepts and principles, and skills were evaluated globally. Recently, competency based approaches have gradually taken over. Although the behaviourist approach may occasionally be used for training in areas where rigid protocols exist, such as the advanced life support course, holistic varieties of the competency based approach are used more widely. In Australia, criterion referenced procedures to set standards have been used to define and measure competency for the graduate entry medical programme.¹⁶ In the United Kingdom, the Royal College of General Practitioners distinguishes between clinical competence (what doctors can do) and clinical performance (what doctors do do) and defines competencies as a combination of knowledge, skills, and attitudes which, when applied to a particular situation, lead to a given outcome. Competency based medical training is usually developed in four steps: determine what the appropriate competencies are, devise training programmes, devise appropriate assessment methods, and set minimum pass standards.

Appropriate competencies can be determined in several ways, such as the GMC's Good Medical Practice for its performance procedures, postal questionnaire surveys of examiners and the committee of trainee members for the part 2 of the examination for membership of the Faculty of Public Health Medicine.¹⁷ Competencies for general practitioners have been defined by using triangulation of results from focus groups with general practitioners, behavioural coding of general practitioners' consultations with patients, and interviews with patients.¹⁸ There is little evidence, however, that addressing each of these competencies separately is a more effective form of training and assessments than the traditional global approach.

(Credit: GEOFF TOMPKINSON/SPL) Competency based medical training is prominent in undergraduate medical education

Based on the competency approach, the objective structured clinical examination using checklists and standardised patients was initially thought to be more reliable and objective and gradually replaced the traditional long case. A recent review has found, however, that, for equal testing time, it is slightly less reliable than the long case.¹⁹ Several possible reasons for this surprising finding were given: standardisation of what happens within a case does not eliminate the variability of performance across clinical problems, and the use of ratings in long cases may achieve higher reliability than checklists. Perhaps another reason is that checklists including attributes such as attitudes and personal values may achieve lower reliability than behavioural outcomes. If this were the case, the exclusive focus on outputs that is often perceived to be the key advantage of the competency based approach does not necessarily result in objective and reliable assessments. In their summary assessments, general practice registrars need to submit a video of seven consultations to demonstrate each prescribed competency at least four times. Some candidates find such an exercise exceedingly time consuming and think that it might hinder other educational opportunities and enjoyment of general practice.²⁰

Leading royal colleges set criterion referenced minimum pass standards by a panel agreeing on the probable scores of borderline candidates for both the written examinations and the objective structured clinical examination. ^{21 22} Although these procedures can be used to set standards for excellence, they currently tend to focus on the minimum acceptable standards. In other examinations and assessments, the pass standards may be more arbitrary.

Other issues are important. Firstly, a key advantage of the competency approach is its focus on competencies achieved rather than time served, so that trainees can progress at their own pace. But the training period for undergraduate and postgraduate medical training is currently fixed. Secondly, the competency approach ignores the learning process, although the process is important for lifelong learning. Thirdly, with the focus of the competency approach on skills and attitudes rather than a solid understanding of the basic concepts and principles, the risk is that "medical education" may give way to "medical training."

	An evaluation

Top The birth of the... The rise of "holistic"... Current scene in medical... An evaluation References

Compared with the traditional approach, the competency based approach potentially leads to individualised flexible training, transparent standards, and increased public accountability. If applied inappropriately, it can also result in demotivation, focus on minimum acceptable standards, increased administrative burden, and a reduction in the educational content. Higher order competencies need to be defined and developed more robustly. We should be cautious of adopting the competency based approach universally across stages of medical training for which well defined and validated competencies are unavailable. After all, it is just one of many potentially useful approaches that may have a role at various stages of the educational progress.

	Acknowledgments

Contributors: WCL is the sole author and guarantor of the paper.

	Footnotes

Competing interests: None declared.

	References

Top The birth of the... The rise of "holistic"... Current scene in medical... An evaluation References

1.	Southgate L, Campbell M, Cox J, Foulkes J, Jolly B, McCrorie P, et al. The General Medical Council's performance procedures: the development and implementation of tests of competence with examples from general practice. Med Educ 2001; 35 (suppl 1): 20-28.
2.	Patterson F, Lane O, Ferguson E, Norfolk T. Competency based selection system for general practitioner registrars. BMJ 2001; 323 (suppl): S2. http://bmj.com/cgi/content/full/323/7311/S2-7311 (accessed 28 Jun 202).
3.	Wood LEP, O'Donnell E. Assessment of competence and performance at interview. BMJ 2000; 320 (suppl): S2. http://bmj.com/cgi/content/full/320/7231/S2-7231 (accessed 28 Jun 2002).
4.	Department of Education. Working together: education and training. London: HMSO, 1986.
5.	Grant G. On competence: a critical analysis of competence-based reforms in higher education. San Francisco: Jossey-Bass, 1975.
6.	Wolf A. Competence-based assessment. Buckingham: Open University Press, 1995.
7.	Mansfield B. Competence and standards. In: Burke JW, ed. Competency based education and training. Lewes: Falmer Press, 1989.
8.	Mitchell L, Wolf A. Understanding the place of knowledge and understanding in a competence based approach. In: Fennel E, ed. Development of assemble standards for national certification. Sheffield: Employment Department, 1991:25-29.
9.	Hyland T. Competence, education and NVQs: dissenting perspectives. London: Cassell, 1994.
10.	Marshall K. NVQs: An assessment of the "outcomes" approach in education and training. J Further Higher Educ 1991; 15: 56-64.
11.	Bates I. The competence and outcomes movement. In: Flude M, Sieminski S, eds. Education, training and the future of work II. London: Routledge, 1999:98-123.
12.	Leung WC. Managers and professionals: competing ideologies. BMJ 2000; 321 (suppl): S2. http://bmj.com/cgi/content/full/321/7266/S2-7266 (accessed 28 Jun 2002).
13.	Gonczi A. Future directions for vocational education in Australian secondary schools. Austral N Z J Vocational Educ Res 1997; 5: 77-108.
14.	Toohey S, Ryan G, Mclean J, Hughes C. Assessing competency-based education and training. Austral N Z J Vocational Educ Res 1995; 3: 86-117.
15.	Fleming D. The concept of meta-competence. Competence Assessment 1993; 22: 6-9.
16.	Searle J. Defining competencythe role of standard setting. Med Educ 2000; 34: 363-366[CrossRef][Web of Science][Medline].
17.	Kisely SR, Donnan SP. Competencies for part 2 of the examination for membership of the Faculty of Public Health Medicine. J Public Health Med 1997; 19: 11-17[Abstract/Free Full Text].
18.	Patterson F, Ferguson E, Lane P, Farrell K, Martlew J, Wells A. A competency model for general practice: implications for selection, training, and development. Br J Gen Pract 2000; 50: 188-193[Web of Science][Medline].
19.	Norman G. The long case versus objective structured examinations. BMJ 2002; 324: 748-749[Free Full Text].
20.	Bahrami J. The lost video. BMJ 2001; 323: 581[Free Full Text].
21.	Angoff WH. Scales, norms and equivalent scores. In: Thorndike RL, ed. Educational measurement. Washington DC: American Council on Education, 1971.
22.	Rothman AI, Cohen R. A comparison of empiricallyand rationallydefined standards for clinical skills checklists. Acad Med 1996; 71 (suppl): S1-S3[CrossRef].

(Accepted 23 May 2002)

Commentary: The baby is thrown out with the bathwater

Vin Diwakar, consultant in general paediatrics and medical education. 

The Education Centre, Birmingham Children's Hospital, Birmingham B4 6NH

vinod.diwakar{at}bhamchildrens.wmids.nhs.uk

Controversy over the competency based approach to professional education centres on a lack of consensus over what the term means.¹

Leung casts doubt on the value of the competency based approach. He takes a narrow view, dismissing work which develops the concept to reflect the complexity of professional practice. Leung ignores evidence and consensus that knowledge driven traditional models of professional training fail to meet the demands of daily practice. ^{2 3}

"Competency" describes what a doctor should be capable of doing, and Leung is correct that education focused entirely on narrow definitions of competencies has limitations for professionals. Reflective practice is ignored by reducing professional practice to an exhaustive list of competencies.⁴

Both traditional medical teaching and the reductionist approach to competence assume that medical education is only about teaching doctors to solve predictable problems. Professional practice requires an education which recognises that patients are treated as individuals. Clinical problems are personal and unique. To solve them, we make informed, but ultimately value based, judgments that are founded on intelligent reflection on previous experience (expertise).

Analysis of the ability of professionals to choose, develop, and adapt abilities for different situations bridges the gap between traditional or reductionist approaches and the realities of practice. Leung dismisses a significant body of work on assessment of these "higher order competencies" or "meta-competencies."

Miller described a four stage hierarchy of competencies, starting with "knowledge," progressing through "know how" and "show how" (competence), and culminating in "does" (performance).⁵ Performance depends on the context in which a doctor works as well as his or her abilities.

Unlike Leung, I think that most professional bodies recognise this hierarchy of professional competence. Methods of assessment change as doctors progress. Certification of medical students and junior trainees is like a driving licence. The minimum that a doctor must be able to do before he or she can move on to the next stage of professional practice and training is specified. Knowledge and competencies are emphasised, but flexibility in thought and action is required.² Certification of senior trainees and reaccreditation of established practitioners focuses on performance. Attempts to define competencies and meta-competencies across the scope of professional practice are likely to be impossible. Thus, assessment makes use of portfolios, peer and self assessment, and clinical outcomes.⁶

Leung's misgivings about competency based education represent one end of a spectrum of views about the extent to which the term includes concepts of competency, meta-competency, and performance. Some argue that the constructivist nature of meta-competency cannot be reconciled to the reductionist industrial origins of the term "competency."¹ Others argue that competency based approaches include elements of all these concepts. It is not surprising that evidence for benefits of the "competency based approach" is hard to find, and disagreement exists over what the terms actually mean. Even so, a recent systematic review found studies showing improved performance by doctors and safety of patients from residents who had attended courses based on competencies. ^{7 8}

In practice, terms are less important than what we do with the concepts that they represent. Several issues are clear. Traditional models of medical education have been found wanting. A sophisticated model of professional education is required that recognises both basic standards and continuing professional development. The best methods of teaching and assessing these components of daily clinical practice need to be established. A fruitless debate about the meaning of "competency based education" is likely to detract from these, the real challenges of the next decade.

	Footnotes

Competing interests: VD holds grants from the Royal College of Paediatrics and Child Health and West Midlands Postgraduate Deanery to develop a competency based curriculum for senior house officers in paediatrics.

	References

1.	Tarrant J. What is wrong with competence? J Further Higher Educ 2000; 24: 77-83[CrossRef].
2.	General Medical Council. Tomorrow's doctors: recommendations on undergraduate medical education. London: GMC, 1993.
3.	Hutchinson L, Aitken P, Hayes T. Are medical postgraduate certification processes valid? A systematic review of published evidence. Med Educ 2002; 36: 73-91[CrossRef][Web of Science][Medline].
4.	Barnett R. The limits of competence: knowledge, higher education and society. Buckingham: Society for Research into Education, 1994.
5.	Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65: 563-567.
6.	Southgate L, Hays RB, Norcini J, Mulholland H, Ayers B, Woolliscroft J, et al. Setting performance standards for medical practice: a theoretical framework. Med Educ 2001; 35: 474-481[CrossRef][Web of Science][Medline].
7.	Caraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med 2002; 77: 361-367[Web of Science][Medline].
8.	Martin M, Vashisht B, Frezza E. Competency based instruction in critical invasive skills improves both resident performance and patient safety. Surgery 1998; 124: 313-317[Web of Science][Medline].