Papers

Clinical experience, performance in final examinations, and learning style in medical students: prospective study

^a Academic Department of Psychiatry, Imperial College School of Medicine at St Mary’s, London W2 1PG, ^b Northwick Park and St Mark’s Trust, Northwick Park Hospital, Harrow HA1 3UJ

Correspondence to: Professor I C McManus Centre for Health Informatics and Multiprofessional Education, University College London Medical School, London N19 5NF i.mcmanus@ucl.ac.uk

	Abstract

Top Abstract Introduction Subjects and methods Results Discussion References

Objective: To assess whether the clinical experience of undergraduate medical students relates to their performance in final examinations and whether learning styles relate either to final examination performance or to the extent of clinical experience.
Design: Prospective, longitudinal study of two cohorts of medical students assessed by questionnaire at time of application to medical school and by questionnaire and university examination at the end of their final clinical year.
Subjects: Two cohorts of students who had applied to St Mary’s Hospital Medical School during 1980 (n=1478) and 1985 (n=2399) for admission in 1981 and 1986 respectively. Students in these cohorts who entered any medical school in the United Kingdom were followed up in their final clinical year in 1986-7 and 1991-2.
Main outcome measures: Students’ clinical experience of a range of acute medical conditions, surgical operations, and practical procedures as assessed by questionnaire in the final year, and final examination results for the students taking their examinations at the University of London.
Results: Success in the final examination was not related to a student’s clinical experiences. The amount of knowledge gained from clinical experience was, however, related to strategic and deep learning styles both in the final year and also at the time of application, five or six years earlier. Grades in A level examinations did not relate either to study habits or to clinical experience. Success in the final examination was also related to a strategic or deep learning style in the final year (although not at time of entry to medical school).
Conclusions: The lack of correlation between examination performance and clinical experience calls into question the validity of final examinations. How much knowledge is gained from clinical experience as a student is able to be predicted from measures of study habits made at the time of application to medical school, some six years earlier, although not from results of A level examinations. Medical schools wishing to select students who will gain the most knowledge from clinical experience cannot use the results of A level examinations alone but could assess a student’s learning style.

	Introduction

Top Abstract Introduction Subjects and methods Results Discussion References

"To study the phenomena of disease without books is to sail an uncharted sea, while to study books without patients is not to go to sea at all." Sir William Osler¹

Medical training—and particularly British medical training^{2 3 4 5}—has always emphasised not merely learning from textbooks but also gaining experience directly from patients themselves. Although clinical experience is perceived as central to medical education, there has been little assessment of how and why medical students differ in the knowledge they gain from clinical experience and how clinical experience relates to success in clinical examinations.⁶ Educational theory predicts that clinical experience and examination success should both relate to study habits.⁷ There are large variations between students in the amount and range of their experience,^{8 9} and differences also occur in postgraduate training.^{10 11} Comparison of entry cohorts of medical students in 1981 and 1986 in their final year of undergraduate training in the United Kingdom confirmed that there were large individual differences in clinical experience as well as secular trends and regional differences.¹²

A levels and other secondary school examinations are comparatively poor predictors of students’ success at university.¹³ Better predictors are students’ approaches to their work, or their study habits and learning styles.^{14 15} The skills needed to do well at A level are less sophisticated than the critical analytical skills that universities instil, and the higher correlation between entry qualifications and final grades in science and medicine than in other university subjects¹³ suggests an undue emphasis on factual recall rather than deeper cognitive analysis. Universities generally value deep learning (table 1) that is motivated by a desire for personal understanding and vocational relevance and demonstrated by the student’s searching for principles and the integration of knowledge across different domains. In contrast, surface learning is motivated principally by fear of failure and is dominated by rote learning for regurgitation in examinations and is followed by forgetting. Strategic learning is motivated by a desire for success and by competition with other students. In strategic learning students use whichever method—deep or surface learning—is more appropriate for a particular topic; the result is patchy understanding and a lack of integration across topics. Empirically, deep and strategic learning styles predict success in final examinations at university, whereas surface learning predicts failure.^{15 16}

This paper assesses the relation between clinical experience, examination success, and study habits in two cohorts of medical students.

	Subjects and methods

Top Abstract Introduction Subjects and methods Results Discussion References

Study design—This study began as two surveys of medical student selection and assessed applicants who had applied for admission to St Mary’s Hospital Medical School in 1981¹⁷ and 1986 (fig 1).¹⁸ All applicants with addresses in the United Kingdom in the first study and addresses in the European Community in the second study were sent a questionnaire which included measures of study habits; there was a response rate of 85% (1151/1362) in the first study and 92% (2043/2209) in the second. Of the 1478 applicants in the first study, 517 were admitted to medical schools in the United Kingdom. Of the 2399 applicants in the second study, 871 were admitted to medical schools in the United Kingdom. In 1986-7, 463 final year students from the first cohort were sent a second questionnaire, as were 761 from the second in 1991-2; the questionnaire measured study habits and clinical experience, and the response rate was 65% (301/461) for the first study and 50% (383/766) for the second.

View larger version (30K): [in this window] [in a new window]   Fig 1 Study design for survey of learning styles and clinical experience in two cohorts of medical students in 1981 (1986)

Study habits and learning styles—Members of the two cohorts received an abbreviated 18 item study process questionnaire developed by Biggs^{19 20 21 22} which assesses learning styles on surface, deep, and strategic scales.⁷ Reliability coefficients () were 0.535, 0.737, and 0.703 in final year students in the first study and 0.567, 0.715, and 0.701 for final year students in the second study. For applicants in 1986 the reliability coefficients were 0.556, 0.713, and 0.647. Applicants in 1981 were assessed on the less reliable measure of syllabus-boundness²³ (=0.497).

Performance in final examinations—Altogether, 330 of the students in the first cohort and 361 of those in the second took final examinations at the University of London. Detailed information was obtained from records in the faculty of medicine. The examination had five separate sections (medicine, which included psychiatry, paediatrics, and public health; surgery; pathology; clinical pharmacology; and obstetrics and gynaecology), all of which had multiple choice and essay questions. All subjects except pathology had an oral examination (viva voce), and all except clinical pharmacology had a practical or clinical examination or both. Principal component analysis, based on a candidate’s individual scores, was used to calculate a single measure of performance, the overall finals score (first cohort: =0.879; second cohort: =0.868).²⁴ Additionally, scores were calculated for the five separate subjects and the four modes of assessment (multiple choice question, essay, oral, and clinical and practical examinations).²⁴

Clinical experience—Students in the first cohort reported their experience of 15 acute medical conditions and those in the second cohort of 20 conditions. Students in each cohort reported their experience of 18 surgical operations. Students in the first cohort reported their experience of 17 practical procedures, and those in the second cohort reported their experience of 29 practical procedures. Scores on the 50 items in the first cohort and 67 in the second were combined into a single total experience score (first cohort: =0.861; second cohort: =0.907).¹²

	Results

Top Abstract Introduction Subjects and methods Results Discussion References

Clinical experience and final examination performance
The correlation between overall performance in final examinations and total clinical experience was not significant (first cohort: r=0.048, P=0.48, n=215; second cohort: r=-0.024, P=0.75, n=176) (fig 2). There was no evidence of specific associations. There were non-significant correlations between experience of treating medical conditions and performance on the medicine section of the final examination (first cohort: r=-0.041, P=0.55; second cohort: r=-0.059, P=0.44); between surgical operations seen and performance in the surgery examination (first cohort: r=0.088, P=0.20; second cohort: r=-0.118, P=0.12); and between overall experience and performance in clinical examinations (first cohort: r=0.026, P=0.70; second cohort: r=-0.042, P=0.58). These results suggest that clinical experience does not influence performance in final examinations.

View larger version (33K): [in this window] [in a new window]   Fig 2 Association between performance in final examination and total amount of clinical experience in 1986 cohort (r=-0.024, P=0.75). The measure of clinical experience12 comprised 20 items scored 1 to 3 and 47 items scored 1 to 4 (total possible range 67 to 230). Performance in the final examination was the principal component of the examinations, scaled to have a mean of 100 and a standard deviation of 15

Study habits and performance in final examination
Study habits in the final year predicted overall examination performance; the correlations with surface, deep, and strategic learning in the first cohort (n=213) were r=-0.204 (P=0.003), r= 0.157 (P=0.022), and r=0.178 (P=0.009) respectively and in the second cohort (n=171) r=-0.081 (P=0.28), r=0.235 (P=0.002), and r=0.266 (P<0.001). The negative associations with surface learning and positive associations with deep and strategic learning were in the expected directions. The pattern was similar for different subjects and examination methods. Although study habits in the final year predicted performance in the final examination in the 1986 cohort (n=344) there were non-significant correlations between examination performance and surface, deep, and strategic processing as assessed at the time of application (r=-0.068, P=0.21; r=0.031, P=0.57; r=-0.004, P=0.94). Similarly, syllabus-boundness at time of application in the 1981 cohort (n=285) showed a non-significant correlation with examination performance (r=-0.007, P=0.90).

Measures of surface, deep, and strategic learning at application and in the final year were correlated in the 1986 cohort (n=361) (r=0.420, r=0.370, r= 0.336 (all P<.001)), confirming moderate long term trait stability. Syllabus-boundness at time of application in the 1981 cohort (n=307) correlated with final year surface, deep, and strategic learning (r=0.205, P<0.001; r=-0.175, P=0.002 ; r=-0.039, P=0.49), confirming construct overlap of syllabus-boundness and learning style.

Study habits and clinical experience
Table 2 shows the correlation of clinical experience and study habits assessed both in the final year and at the time of application. Students with higher scores on deep and strategic learning show significantly higher levels of overall experience (fig 3) whether study habits are measured in the final year or at the time of selection—five or six years earlier. Surface learning (and syllabus-boundness) scores at application showed significant negative correlations with clinical experience.

View larger version (41K): [in this window] [in a new window]   Fig 3 Association between total amount of clinical experience and score on Biggs’s measure of deep learning in applicants in the 1986 cohort (r=0.262, P<0.001). Clinical experience measured as in fig 2

A level examinations
Grades at A level are a potential confounder of the associations reported here. Mean A level grade^{17 18} showed a significant correlation with performance in final examinations (first cohort: r=0.336, P<0.001, n=329; second cohort: r=0.281, P<0.001, n=359); these values are compatible with those reported elsewhere.^{25 26} However, grades at A level showed almost no association with syllabus-boundness in the 1981 cohort (r=0.047, P=0.12, n=1104) and with surface, deep, and strategic learning at the time of selection in the second cohort (r=-0.021, P=0.34; r=-0.051, P=0.024; r=0.024, P=0.29; n=1932, NS). There was no correlation with learning styles in the final year (first cohort (n=333): r=-0.076, P=0.26 for surface learning; r=0.080, P=0.15 for deep learning; r=0.099, P=0.07 for strategic learning; second cohort (n=371): r=0.091, P=0.080; r=-0.005, P=0.93; r=0.003, P=0.96); grades at A level did not correlate with clinical experience (first cohort: r=0.023, P=0.68, n=335; second cohort: r=-0.034, P=0.51, n=370).

Non-respondents
A potential risk in a study in which only 51% of students in the first cohort and 65% in the second replied to the questionnaire during their final year is that respondents may represent a biased subset of those in the initial study. In the initial study, however, this bias is unlikely to be a serious problem since response rates were satisfactory at 85% and 93% respectively. We assessed possible bias by comparing baseline measures and the final examination performance of those who did and those who did not return our questionnaire during their final year. In neither cohort were there significant differences between respondents and non-respondents in the final year in terms of study habits at the time of selection, final year examination performance, or in mean grade at O level, and number of O and A levels taken. In the 1981 cohort the non-respondents had slightly lower mean grades at A level (3.81 (SD 0.77), n=177 v 3.98 (0.78), t=2.44, df=513, P=0.015); the effect was not significant in the 1986 cohort (4.16 (0.741), n=377 v 4.19 (0.64), t=0.67, df=513, P=0.051). Our respondents were probably a representative sample of the students as a whole.

	Discussion

Top Abstract Introduction Subjects and methods Results Discussion References

Medical students work hard to acquire clinical experience.²⁷ If clinical problem solving is the key to learning for medical students and doctors,²⁸ potential doctors should see many patients and medical students with clinical experience should be seen to benefit from it, not least by performing better in clinical examinations. If clinical experience is educationally desirable then students who are likely to learn most from their contact with patients should be selected to study medicine.

Validity of final examinations
Our study shows that students with more clinical experience do not do better in final examinations either generally or specifically in the clinical sections of examinations. This conclusion is robust, being found in two cohorts of reasonable size studied five years apart. We do not believe our results reflect any specific failing of the University of London’s examinations, which are typical of those in most medical schools in the United Kingdom and have external examiners at all levels. We also recognise that the radical educational and curricular changes being introduced into medical schools since the publication of Tomorrow’s Doctors by the General Medical Council²⁹ may invalidate our findings for future generations of medical students. That, however, is an empirically testable hypothesis, and our current study provides the baseline data needed for assessing the reforms.

The lack of correlation between the amount of clinical experience and performance in final examinations calls into question the clinical validity of the examinations; our conclusions may also apply to similarly structured postgraduate examinations. Examinations may be failing to assess the skills and knowledge acquired directly from clinical experience, such as carrying out practical procedures, communicating with patients, formulating differential diagnoses, ordering tests, evaluating their results, and deciding on management. An alternative possibility is that students are failing to learn properly from their experience since "[clinical] experience without training increases confidence not competence."^{30 31} Whatever the mechanism, it must be a concern that examinations send the wrong messages to undergraduates. Assessments determine what and how students choose to study. If skills and experiences acquired from patients are not seen to be relevant to success in examinations then fear of failure will drive some students to ignore clinical experience and resort instead to what is perceived as the real curriculum—that is, the information contained in textbooks. These attitudes seem unlikely to encourage the lifelong learning necessary for doctors, or to foster the development of "reflective practitioner[s]"³² who can modify their practice in relation to experience.

Since strategic and deep learning styles correlate positively with performance in final examinations and surface learning correlates negatively, the present examinations are probably encouraging a deeper understanding of medicine and medical practice. That other studies have failed to find this probably indicates that they had too small a sample size.³³ That study habits at the time of selection for admission to medical school did not predict results in final examinations reflects the fact that study habits are states as much as traits. There is a well documented trend towards surface learning in conventional medical education,^{34 35} a process driven in part by failure in previous examinations.³⁶ Courses structured less conventionally, perhaps using problem based learning,^{34 35} may find a correlation between study habits at entry and performance in final examinations.

Clinical experience and study habits
An important finding of this study is that the knowledge acquired by a clinical student can be predicted from the learning style measured at the time of application to medical school—half a decade earlier. This implies that the acquisition of knowledge from clinical experience—and the ability to continue to gain experience throughout a professional career—is a characteristic that can be identified at the time of selection. However, selection of medical students is based primarily on the results of examinations that do not correlate with the learning styles that are desirable in medical students, and these examinations do not predict the successful acquisition of clinical experience. The implication is that the greater the dependence of a selection system on grades at A level the more limited is its capacity for selecting doctors who will gain the most knowledge from clinical experience and therefore probably continue to benefit from clinical experience throughout their careers.²⁹ Evaluation of the effectiveness of medical training should concentrate on characteristics other than the ability to pass examinations, both as an input measure for selection and as an output measure of the quality of medical education.

	Acknowledgements

We thank the Faculty of Medicine of the University of London for permission to analyse the examination results of undergraduates, and we thank our many respondents for completing our questionnaires.

Funding: The survey of the 1981 cohort was funded by the Economic and Social Research Council, and the survey of the 1986 cohort by the Economic and Social Research Council and the Leverhulme Trust.

Conflict of interest: None.

	Notes

Contributors: ICM and PR initiated and designed the cohort studies; ICM was responsible for management of the studies and for statistical analysis; BCW coordinated the acquisition and processing of the data; KAS and BCW collected much of the data; the manuscript was written by ICM and PR with help from BCW and KAS. ICM is guarantor for this paper.

	References

Top Abstract Introduction Subjects and methods Results Discussion References

1. Salter RH. Stop denigrating service. BMJ 1995;310:538. [Free Full Text]
2. Flexner A. Medical education: a comparative study. New York: MacMillan, 1925.
3. Pickering G. Quest for excellence in medical education: a personal survey. Oxford: Oxford University Press, 1978.
4. Sinclair D. Basic medical education. London: Oxford University Press, 1972.
5. Bonner TN. Becoming a physician: medical education in Great Britain, France, Germany, and the United States, 1750-1945. New York: Oxford University Press, 1995.
6. Jolly BC, Jones A, Dacre JE, Elzubeir M, Kopelman P, Hitman G. Relationships between students’ clinical experiences in introductory clinical courses and their performances on an objective structured clinical examination (OSCE). Acad Med 1996;71:909-916. [Medline]
7. Newble DI, Entwistle NJ. Learning styles and approaches: implications for medical education. Med Educ 1986;20:162-175. [Medline]
8. Lockwood DNJ, Goldman LH, McManus IC. Clinical experience of clerks and dressers: a three-year study of Birmingham medical students. J R Soc Med 1986;79:38-42. [Abstract]
9. Lockwood DNJ, Goldman LH, McManus IC. Surgical dressers: the theatre experience of junior clinical students. Med Educ 1986;20:216-21. [Medline]
10. Dent THS, Gillard JH, Aarons EJ, Crimlisk HL, Smyth-Piggott PJ. Preregistration house officers in the four Thames regions. I. Survey of education and workload. II. Comparison of education and workload in teaching and non-teaching hospitals. BMJ 1990;300:713-8.
11. Langdorf MI, Strange G, Macneil P. Computerized tracking of emergency medicine resident clinical experience. Ann Emerg Med 1990;19:764-73. [Medline]
12. McManus IC, Richards P, Winder BC, Sproston KA, Vincent CA. The changing clinical experience of British medical students. Lancet 1993;341:941-4. [Medline]
13. Peers IS, Johnston M. Influence of learning context on the relationship between A-level attainment and final degree performance: a meta-analytic review. Br J Educ Psychol 1994;64:1-17.
14. Entwistle N. Styles of learning and teaching. Chichester: Wiley, 1981.
15. Ramsden P, Entwistle NJ. Effects of academic departments on students’ approaches to studying. . Br J Educ Psychol 1981;51:368-83.
16. Kleijn WC, van der Ploeg HM, Topman RM. Cognition, study habits, test anxiety, and academic performance. Psychol Rep 1994;75:1219-26. [Medline]
17. McManus IC, Richards P. An audit of admission to medical school. I. Acceptances and rejects. BMJ 1984;289:1201-4.
18. McManus IC, Richards P, Maitlis SL. Prospective study of the disadvantage of people from ethnic minority groups applying to medical schools in the United Kingdom. BMJ 1989;298:723-6.
19. Biggs JB. Study process questionnaire: manual. Melbourne: Australian Council for Educational Research, 1987.
20. Biggs JB. Student approaches to learning and studying. Melbourne: Australian Council for Educational Research, 1987.
21. O’Neil MJ, Child D. Biggs’ SPQ: a British study of its internal structure. Br J Educ Psychol 1984;54:228-34.
22. Biggs JB. What do inventories of students’ learning processes really measure? A theoretical review and clarification. Br J Educ Psychol 1993;63:3-19.
23. Lucas CJ, Crown S, Stringer P, Supramaniam S. Further observation on study difficulty in university students including "syllabus-boundness." Br J Psychiatry 1976;129:598-603. [Abstract/Free Full Text]
24. McManus IC, Richards P, Winder BC, Sproston KA. Final examination performance of students from ethnic minorities. Med Educ 1996;30:195-200. [Medline]
25. Tomlinson RWS, Clack GB, Pettingale KW, Anderson J, Ryan KC. The relative role of ‘A’ level chemistry, physics and biology in the medical course. Med Educ 1977;11:103-8. [Medline]
26. McManus IC, Tunnicliffe N, Fleming PR. The independent effects of intelligence and educational achievements in predicting final examination success. Med Educ 1990;24:181-4. [Medline]
27. Atkinson P. The clinical experience: the construction and reconstruction of medical reality. Farnborough: Gower, 1981.
28. Slotnick HB. How doctors learn: the role of clinical problems across the medical school-to-practice continuum. Acad Med 1996;71:28-34. [Medline]
29. General Medical Council. Tomorrow’s doctors: recommendations on undergraduate medical education. London: GMC, 1993.
30. Marteau TM, Wynne G, Kaye W, Evans TR. Resuscitation: experience without feedback increases confidence but not skill. BMJ 1990;300:849-50.
31. Bulstrode C, Holsgrove G. Education for educating surgeons. BMJ 1996;312:326-7. [Free Full Text]
32. Schön DA. The reflective practitioner. New York: Basic Books, 1983.
33. Leiden LI, Crosby RD, Follmer H. Assessing learning-style inventories and how well they predict academic performance. Acad Med 1990;65:395-401. [Medline]
34. Coles CR. Differences between conventional and problem-based curricula in their students’ approaches to studying. Med Educ 1985;19:308-9. [Medline]
35. Mårtenson DF. Students’ approaches to studying in four medical schools. Med Educ 1986;20:532-4. [Medline]
36. Tooth D, Tonge K, McManus IC. Anxiety and study methods in pre-clinical students: causal relation to examination performance. Med Educ 1989;23:416-21. [Medline]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    Twitter    What's this?

Relevant Article

Clinical experience and performance in final examinations

Wai-Ching Leung, Brian Jolly, Elizabeth Murray, Chris McManus, Belinda Winder, and Peter Richards
BMJ 1998 316: 1984. [Extract] [Full Text]

This article has been cited by other articles:

• Crede, M., Kuncel, N. R. (2008). Study Habits, Skills, and Attitudes: The Third Pillar Supporting Collegiate Academic Performance. Perspectives on Psychological Science 3: 425-453 [Abstract] [Full text]  
• Walsh, K. (2005). How to pass exams: evidence-based advice?. JRSM 98: 294-294 [Full text]  
• Carney, P. A., Nierenberg, D. W., Pipas, C. F., Brooks, W. B., Stukel, T. A., Keller, A. M. (2004). Educational Epidemiology: Applying Population-Based Design and Analytic Approaches to Study Medical Education. JAMA 292: 1044-1050 [Abstract] [Full text]  
• Blair, M. (2003). Training and Education as a Means of Increasing Equity in Child Health Teaching of Undergraduates. Pediatrics 112: 747-748 [Abstract] [Full text]  
• McManus, I C, Smithers, E., Partridge, P., Keeling, A, Fleming, P. R (2003). A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study. BMJ 327: 139-142 [Abstract] [Full text]  
• Wass, V., Roberts, C., Hoogenboom, R., Jones, R., Van der Vleuten, C. (2003). Effect of ethnicity on performance in a final objective structured clinical examination: qualitative and quantitative study. BMJ 326: 800-803 [Abstract] [Full text]  
• McLaughlin, R, Bell, R (2002). Teaching and learning. Emerg. Med. J. 19: 375-376 [Full text]  
• Dacre, J. E, Fox, R. A (2000). Series on education: How should we be teaching our undergraduates?. Ann Rheum Dis 59: 662-667 [Full text]  
• McManus, I C, Richards, P, Winder, B C (1999). Intercalated degrees, learning styles, and career preferences: prospective longitudinal study of UK medical students. BMJ 319: 542-546 [Abstract] [Full text]  
• Leung, W.-C., Jolly, B., Murray, E., McManus, C., Winder, B., Richards, P. (1998). Clinical experience and performance in final examinations. BMJ 316: 1984-1984 [Full text]