Learning needs assessment: assessing the need
BMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7330.156 (Published 19 January 2002) Cite this as: BMJ 2002;324:156- Janet Grant, professor of education in medicine (j.r.grant{at}open.ac.uk)
- Accepted 17 December 2001
Editorial by Goldbeck-Wood and Peile
Learning needs assessment has a fundamental role in education and training, but care is needed to prevent it becoming a straitjacket
It might seem self evident that the need to learn should underpin any educational system. Indeed, the literature suggests that, at least in relation to continuing professional development, learning is more likely to lead to change in practice when needs assessment has been conducted, the education is linked to practice, personal incentive drives the educational effort, and there is some reinforcement of the learning.1 Learning needs assessment is thus crucial in the educational process, but perhaps more of this already occurs in medical education than we suspect. The key lesson might be for those who design new systems of education and training: for example, the postgraduate education allowance system in general practice was felt to fail the profession because it did not include needs assessment and so led to ad hoc education to fulfil the time requirements of the system rather than the needs of individual doctors or the profession as a whole. On the other hand, basing learning in a profession entirely on the assessment of needs is a dangerous and limiting tactic. So a balance must be struck.
Summary points
Learning needs assessment is a crucial stage in the educational process that leads to changes in practice, and has become part of government policy for continuing professional development
Learning needs assessment can be undertaken for many reasons, so its purpose should be defined and should determine the method used and the use made of findings
Exclusive reliance on formal needs assessment could render education an instrumental and narrow process rather than a creative, professional one
Different learning methods tend to suit different doctors and different identified learning needs
Doctors already use a wide range of formal and informal ways of identifying their own learning needs as part of their ordinary practice
These should be the starting point in designing formalised educational systems for professional improvement
Learning needs assessment in medicine
In 1998 both individual and organisational needs assessment became part of government policy in relation to the continuing professional development and personal development plans of all healthcare professionals.2 Thus, it has a role in the clinical governance of the service3 and is therefore much more than an educational undertaking. This integration of needs assessment, education, and quality assurance of the service was first made explicit in 1989 in relation to clinical audit, which would identify practices in need of improvement and ensure that educational and organisational interventions were made to address these needs.4 Accordingly, audit was described as “essentially educational” and the educational process surrounding it described.5
Long before these recent developments, needs assessment outside medicine was presented as an important part of managed education and learning contracts, which are the predecessors of the personal development plans to be developed for all NHS healthcare professionals.6 In his descriptions of adult learning Knowles assumed (he did not claim to have research evidence) that learners needed to feel a necessity to learn and that identifying one's own learning needs was an essential part of self directed learning.7 In medicine a doctor's motivation to learn would therefore derive from needs identified during his or her experience of clinical practice. So the pedigree and practice of learning needs assessment, if not the evidence, are well established.
The definition of need
As in most areas of education, for many years there has been intense debate about the definition, purpose, validity, and methods of learning needs assessment.8 It might be to help curriculum planning, diagnose individual problems, assess student learning, demonstrate accountability, improve practice and safety, or offer individual feedback and educational intervention. Published classifications include felt needs (what people say they need), expressed needs (expressed in action) normative needs (defined by experts), and comparative needs (group comparison).9 Other distinctions include individual versus organisational or group needs, clinical versus administrative needs, and subjective versus objectively measured needs.10 The defined purpose of the needs assessment should determine the method used and the use made of findings.
Furthermore, even though the concept of educational needs assessment is enshrined in practice, policy, and the educational canon, several factors indicate the need for careful planning and research in this subject (see boxes 1 and 2). Exclusive reliance on formal needs assessment in educational planning could render education an instrumental and narrow process rather than a creative, professional one. This is especially so in a profession where there is inherent unpredictability and uncertainty. Members of any profession require wide knowledge and depth of experience—the relevance of some of which might not have been obvious at the time of learning. Certainly, learning needs can and should be identified on the basis of what has been experienced and of what more experienced members of the profession know to be relevant, but this must not deter other, more general or even speculative, learning that, at the time, seems to answer no specific need. Possibly no specific learning needs assessment would ever send a person to a large international conference on a generic subject (such as endocrinology, medical education, or management). It is, nevertheless, important that doctors attend such meetings and return with the unexpected and expected benefits that they accrue.
Box 1 : Need for careful planning in needs assessment
There is little evidence that needs assessment alone enhances educational effectiveness and outcomes, so it must be placed within the wider process of planned learning, relevance to practice, and reinforcement of learning in the appropriate context
Formal needs assessment can identify only a narrow range of needs and might miss needs not looked for, so breadth and flexibility of needs assessment methods should be embraced
In professional education it is not necessarily defensible to focus all learning on identified needs—wider professional learning not related to a specific need is also of fundamental value where practice is not predictable
Individual and group learning needs are different—group learning needs may produce an average picture that fails to address important needs and interests of individual members of the group—so a balance is required. Each approach has its uses and effects, but each must be used for the right purpose
Identifying individual learning needs, often not shared by others, may lead to an unimpressive cost-benefit analysis in terms of individually targeted use of educational resources if used inappropriately. Individual learning needs assessment is best used in the context of learning that occurs on an individual basis—such as in the relationship between general practitioner registrar and trainer
Box 2 : Need for research into needs assessment in medical education
What are the effects of and responses to needs assessment alone for students, trainees, and senior doctors at different stages of medical education?
What is the relative validity, reliability, or utility of different formal and informal methods of learning needs assessment in medical education at any level?
To what extent do needs assessment methods identify all important learning needs?
What are the relative effects and efficacy of identifying group and individual learning needs?
What methods of planning effective learning experiences are most effective on the basis of needs identified?
Methods of needs assessment
Although the literature generally reports only on the more formal methods of needs assessment, doctors use a wide range of informal ways of identifying learning needs as part of their ordinary practice. These should not be undervalued simply because they do not resemble research. Questionnaires and structured interviews seem to be the most commonly reported methods of needs assessment, but such methods are also used for evaluation, assessment, management, education, and now appraisal and revalidation.11 Together, these formal and informal methods might make an effective battery where there is clarity of purpose. The Good CPD Guide details 46 formal and informal methods of self assessment (see box 3).12
Box 3 : Good CPD Guide's classification of sources of needs assessment12
Clinician's own experiences in direct patient care
“Blind spots”
Clinically generated unknowns
Competence standards
Diaries
Difficulties arising in practice
Innovations in practice
Knowledgeable patients
Mistakes
Other disciplines
Patients' complaints and feedback
Necropsies and the clinico-pathological conference
PUNs (patient unmet needs) and DENs (doctor's educational needs)13
Reflection on practical experience
Interactions within the clinical team and department
Clinical meetings—department and grand rounds
Department business plan
Department educational meetings
External recruitment
Junior staff
Management roles
Mentoring
Formal approaches to quality management and risk assessment
Audit
Morbidity patterns
Patient adverse events
Patient satisfaction surveys
Risk assessment
Specific activities directed at needs assessment
Clinical incident surveys
Gap analysis
Objective tests of knowledge and skill
Observation
Revalidation systems
Self assessment
Video assessment of performance
Peer review
External
Informal—of the individual doctor
Internal
Multidisciplinary
Physician assessment
Non-clinical activities
Academic activities
Conferences
International visits
Journal articles
Medicolegal cases
Press and media
Professional conversations
Research
Teaching
The methods listed are both formal and informal, planned and opportunistic, showing that day to day work and encounters have the potential to generate needs as much as do formal methods. Formal needs assessment methods include critical incident techniques, gap analysis, objective knowledge and skills tests, observation, revalidation, self assessment, video assessment, and peer review. Such methods are often used to identify group needs. 14 15 Formal identification of needs can also arise from audit, morbidity patterns, adverse events, patient satisfaction surveys, and risk assessment. Most of these tools use quantitative methods that can generate computerised data and cover wider population ranges, but these are often unable to probe into the personal agendas and opinions of individuals.
Types of needs assessment
Methods of needs assessment can be classified into seven main types, each of which can take many different forms in practice.
Gap or discrepancy analysis
This formal method involves comparing performance with stated intended competencies—by self assessment, peer assessment, or objective testing—and planning education accordingly. 9 16 17
Reflection on action and reflection in action
Reflection on action is an aspect of experiential learning and involves thinking back to some performance, with or without triggers (such as videotape or audiotape), and identifying what was done well and what could have been done better. 18 19 The latter category indicates learning needs.
Reflection in action involves thinking about actual performance at the time that it occurs and requires some means of recording identified strengths and weaknesses at the time. The Canadian MOCOMP programme uses formalised reflection as its basic process.20 Similarly, PUNs and DENs (see box 3) are well known in British general practice.
Self assessment by diaries, journals, log books, weekly reviews
This is an extension of reflection that involves keeping a diary or other account of experiences.21 However, practice might show that such documents tend to be written nearer the time of their review than the time of the activity being recorded.
Peer review
This is rapidly becoming a favourite method. It involves doctors assessing each other's practice and giving feedback and perhaps advice about possible education, training, or organisational strategies to improve performance. The Good CPD Guide describes five types of peer review—internal, external, informal, multidisciplinary, and physician assessment.11 The last of these is the most formal, involving rating forms completed by nominated colleagues, and shows encouraging levels of validity, reliability, and acceptability. 22 23
Observation
In more formal settings doctors can be observed performing specific tasks that can be rated by an observer, either according to known criteria or more informally. The results are discussed, and learning needs are identified. The observer can be a peer, a senior, or a disinterested person if the ratings are sufficiently objective or overlap with the observer's area of expertise (such as communication skills or management).
Critical incident review and significant event auditing
Although this technique is usually used to identify the competencies of a profession or for quality assurance, it can also be used on an individual basis to identify learning needs.24 The method involves individuals identifying and recording, say, one incident each week in which they feel they should have performed better, analysing the incident by its setting, exactly what occurred, and the outcome and why it was ineffective.
Practice review
A routine review of notes, charts, prescribing, letters, requests, etc, can identify learning needs, especially if the format of looking at what is satisfactory and what leaves room for improvement is followed.
The difference between needs assessment and assessment
Needs assessment is not the same as assessment in the sense of examination of learning. Assessment systems that lead to academic or professional awards should show certain minimum characteristics, including measurement of performance against external criteria and standards, a decision on adequacy by an assessor, and standardised data gathering.25 Needs assessment might sometimes have these characteristics, but it also might be based on practice, reflection, professional judgment, discussion, and informal data. Needs assessment methods that are limited by the standards of assessment will fall into the trap of assessing only a narrow range of needs.
Learning for needs
The main purpose of needs assessment must be to help educational planning, but this must not lead to too narrow a vision of learning. Learning in a profession is unlike any other kind of learning. Doctors live in a rich learning environment, constantly involved in and surrounded by professional interaction and conversation, educational events, information, and feedback. The search for the one best or “right” way of learning is a hopeless task,1 especially if this is combined with attempting to “measure” observable learning. Research papers show, at best, the complexity of the process.
Multiple interventions targeted at specific behaviour result in positive change in that behaviour.26 Exactly what those interventions are is less important than their multiplicity and targeted nature. On the other hand, different doctors use different learning methods to meet their individual needs. For example, in a study of 366 primary care doctors who identified recent clinical problems for which they needed more knowledge or skill to solve, 55 different learning methods were selected.27 The type of problem turned out to be the major determinant of the learning method chosen, so there may not be one educational solution to identified needs.
Much of doctors' learning is integrated with their practice and arises from it. The style of integrated practice and learning (“situated learning”) develops during the successive stages of medical education.28 The components of apprenticeship learning in postgraduate training are made up of many activities that may be regarded as part of practice (see box 4).29 Senior doctors might also recognise much of their learning in some of these elements and could certainly add more—such as conversations with colleagues.
Box 4 : Components of apprenticeship learning in postgraduate training29
Learning by doing
Experience of seeing patients
Building up personal knowledge and experience
Discussing patients
Managing patients
Having errors corrected
Making teaching points during service
Listening to experts' explanations
“Picking things up”
Charismatic influences
Learning clinical methods from practice
Being questioned about thought and actions about patients
Teaching by doing
Using knowledge and skill
Bite-size learning from “bits and pieces”
Retrieving and applying knowledge stored in memory
Learning from supervision
Receiving feedback
Presentation and summarising
Observing experts working
Learning from role models
Learning from team interactions
Hearing consultants thinking aloud
Thinking about practice and patients
Thus, educational planning on the basis of identified needs faces real challenges in making learning appropriate to and integrated with professional style and practice. The first step in all of this is to recognise the needs assessment and learning that are a part of daily professional life in medicine and to formalise, highlight, and use these as the basis of future recorded needs assessment and subsequent planning and action, as well as integrating them with more formal methods of needs assessment to form a routine part of training, learning, and improving practice.
Footnotes
-
Competing interests None declared.