Published 11 July 2008, doi:10.1136/bmj.39540.415822.AD
Cite this as: BMJ 2008;337:a513

Head to Head

Are learning portfolios worth the effort? Yes

¹ Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands

Portfolios were introduced with the aim of improving the learning and assessment of doctors. Erik Driessen believes that they work well when used correctly, but Geoff Norman (doi: 10.1136/bmj.39541.449306.AD) remains unconvinced

A major challenge facing us today is the move to assess doctors’ performance in the workplace instead of the examination hall. The portfolio remains our best solution. It allows the collation and integration of evidence on competence and performance from different sources to gain a comprehensive picture of everyday practice. Simultaneously, portfolios can guide and coach professional development. Studies in multiple contexts confirm that this is feasible if, and only if, users take on board the conditions required for effective use of portfolios.^{1 2}

Portfolios work

To provide credible evidence of fitness to practise doctors have to show in realistic, often stressful, situations that they are competent in all aspects of patient management, diagnostics, communication, teamwork, administration, and professionalism. Since the 1990s various instruments have been developed to assess workplace based learning: the mini-clinical evaluation exercise, multisource feedback, case based discussions, clinical work sampling, and direct observation of procedural skills.³

These tools provide piecemeal information on performance. None is perfect. A portfolio amalgamates evidence from the different sources, allowing assessors to make an overall judgment of competency. The strengths of one assessment method can compensate for the limitations of another. Recent reviews confirm that portfolios effectively assess day to day performance.^{1 2} A comprehensive range of information, collated in this way, can produce a well founded summative judgment.¹ There is an important additional advantage. Doctors or students can simultaneously analyse their own performance. They can reflect on and improve their practice and set realistic objectives for further learning.^{4 5}

However, when weighing the merits of portfolios, it is essential to realise that there is no one standard portfolio. Portfolios are as diverse as their potential content and can be adapted for various purposes.⁶ Their flexibility is a clear advantage. They can be tailored to the specific objectives or outcomes being assessed. Introducing portfolios is like buying new shoes: one size does not fit all. Careful fitting is essential. Their flexibility becomes a disadvantage when they are not tailored to the objectives they are supposed to help attain. If this is the case, misunderstandings arise.

When are portfolios worth the effort?

Despite many vociferous advocates, portfolios can be unpopular with medical teachers and students. Frequently heard complaints are: "It takes up far too much time," "Why do we have to lug around these useless piles of paper?" and "What on earth am I to put in this self-reflection report?"

What are the facts? Research into their effectiveness is as heterogeneous as the portfolios themselves. Many reports have methodological limitations.¹ However, the studies have several common findings. The literature shows that inappropriate use of portfolios can seriously undermine any potential benefit.^{1 2} Although the portfolio concept may seem deceptively simple, it is only too easy for the desired integrated, comprehensive picture to drown in a disorganised mess of useless information. Fortunately, the literature shows that three simple conditions can prevent this.^{1 2 7}

Mentoring is the single most decisive success factor.^{7 8} Without an audience, every portfolio is arguably a waste of time.⁹ If students or doctors are to remain motivated to collate a portfolio, they need regular meetings with their mentor to reflect on the information, diagnose the state of their competence, and set further learning goals. Evidence shows that portfolios improve the planning and monitoring of continuing medical education by combining external assessment and self assessment with mentoring. They enable the doctor to develop more challenging learning goals than is customary in traditional continuing medical education.^{10 11}

Secondly, the portfolio must be smart and lean.¹ Doctors and students alike have a healthy dislike for messy and massive portfolios.¹² A user friendly portfolio contains well organised materials confined to the portfolio’s purpose. It must be located quickly and with ease. Finally, users must have clear instructions and guidelines.¹³ For many, portfolios are foreign to the educational tradition they are accustomed to. The concept of combining formative professional development alongside summative assessment is new. Clear guidelines on the purpose, contents, and organisation of the portfolio are essential.¹⁴

Careful implementation is crucial. A strong resistance to the portfolio can be unleashed when learners are forced to stick to a rigidly prescribed format.^{4 15} Conversely, when learners are allowed to create a portfolio that reflects their personal interests and concerns, they will have a sense of ownership and be motivated to develop its content.^{7 16}

We need to overcome existing tensions in portfolio design and seek a strong evidence base to optimise their use. With proper mentoring, restricted but relevant content, and well balanced guidelines reflective of its purpose, a portfolio undoubtedly makes an important contribution to the effective assessment, both formative and summative, of performance in the workplace.

Cite this as: BMJ 2008;337:a513

---

Competing interests: None declared.

References

1. Driessen E, Van Tartwijk J, Van der Vleuten C, Wass V. Portfolios in medical education: why do they meet with mixed succcess? A systematic review. Med Educ 2007;41:1224-33.[Web of Science][Medline]
2. McCready T. Portfolios and the assessment of competence in nursing. Int J Nurs Stud 2007;44:143-51.[CrossRef][Web of Science][Medline]
3. Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-96.[Free Full Text]
4. Snadden D, Thomas M. Portfolio learning in general practice vocational training—does it work? Med Educ 1998;32:401-6.[CrossRef][Web of Science][Medline]
5. McMullan M, Endacott R, Gray MA, Jasper M, Miller CML, Scholes J, et al. Portfolios and the assessment of competence: a review of the literature. J Adv Nurs 2003;41:283-94.[CrossRef][Web of Science][Medline]
6. Webb C, Endacott R, Gray M, Jasper M, Miller C, McMullan, et al. Models of portfolios. Med Educ 2002;36:897-8.[CrossRef][Web of Science][Medline]
7. Driessen EW, Van Tartwijk J, Overeem K, Vermunt JD, Van der Vleuten CPM. Conditions for successful reflective use of portfolios in undergraduate medical education. Med Educ 2005;39:1230-5.[CrossRef][Web of Science][Medline]
8. Finlay IG, Maughan TS, Webster DJ. A randomized controlled study of portfolio learning in undergraduate cancer education. Med Educ 1998;32:172-6.[CrossRef][Web of Science][Medline]
9. Pearson DJ, Heywood P. Portfolio use in general practice vocational training: a survey of GP registrars. Med Educ 2004;38:87-95.[CrossRef][Web of Science][Medline]
10. Mathers NJ, Challis MC, Howe AC, Field NJ. Portfolios in continuing medical education—effective and efficient? Med Educ 1999;33:521-30.[CrossRef][Web of Science][Medline]
11. Campbell CM, Parboosingh JT, Gondocz ST, Babitskaya G, Lindsay E, De Guzman RC, et al. Study of physicians’ use of a software program to create a portfolio of their self directed learning. Acad Med 1996;71:S49-50.[Web of Science][Medline]
12. Davis MH, Ben-David M, Harden RM, Howie P, Ker J, McGhee C, et al. Portfolio assessment in medical students’ final examinations. Med Teach 2001;23:357-66.[CrossRef][Web of Science][Medline]
13. Kjaer NK, Maagaard R, Wied S. Using an online portfolio in postgraduate training. Med Teach 2006;28:708-12.[CrossRef][Web of Science][Medline]
14. Duque G, Finkelstein A, Roberts A, Tabatabai D, Gold SL, Winer LR. Learning while evaluating: the use of an electronic evaluation portfolio in a geriatric medicine clerkship. BMC Med Educ 2006;6:4.[CrossRef][Medline]
15. Grant AJ, Vermunt JD, Kinnersley P, Houston H. Exploring students’ perceptions on the use of significant event analysis, as part of a portfolio assessment process in general practice, as a tool for learning how to use reflection in learning. BMC Med Educ 2007;7:5.[CrossRef][Medline]
16. Amsellem-Ouazana D, Van Pee D, Godin V. Use of portfolios as a learning and assessment tool in a surgical practical session of urology during undergraduate medical training. Med Teach 2006;28:356-9.[CrossRef][Web of Science][Medline]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Article

Are learning portfolios worth the effort? No

Geoff Norman
BMJ 2008 337: a514. [Extract] [Full Text]

Rapid Responses:

Read all Rapid Responses

A view on electronic portfolio records in the Mersey region

Ram P Narayanan, et al.
bmj.com, 23 Jul 2008 [Full text]

Online poll

Find out more >>