BMJ No 7113 Volume 315
Papers Saturday 11 October 1997
Can students learn clinical method in general practice? A randomised crossover trial based on objective structured clinical examinations
Elizabeth Murray, Brian Jolly, Michael Modell
Abstract
Objective: To determine whether students acquired clinical skills as well in general practice as in hospital and whether there was any difference in the acquisition of specific skills in the two environments.Design: Randomised crossover trial.
Subjects and setting: Annual intake of first year clinical students at one medical school.
Intervention: A 10 week block of general internal medicine, one half taught in general practice, the other in hospital. Students started at random in one location and crossed over after five weeks.
Outcome measures: Students' performance in two equivalent nine station objective structured clinical examinations administered at the mid and end points of the block: a direct comparison of the two groups' performance at five weeks; analysis of covariance, using their first examination scores as a covariate, to determine students' relative improvement over the second five weeks of their attachment.
Results: 225 students rotated through the block; all took at least one examination and 208 (92%) took both. For the first half of the year there was no significant difference in the students' acquisition of clinical skills in the two environments; later, however, students taught in general practice improved slightly more than those taught in hospital (P=0.007).
Conclusions: Students can learn clinical skills as well in general practice as in hospital; more work is needed to clarify where specific skills, knowledge, and attitudes are best learnt to allow rational planning of the undergraduate curriculum.
Introduction
There is an international move towards community based undergraduate medical education.(1-2) In Britain half of medical schools have some primary care input into teaching clinical skills(3); most new curricula will have a substantial increase in community based teaching. This change reflects the "primary care led NHS"(4) that is having a profound impact on delivery of health care(5) and on undergraduate education. However, community based teaching is no cheaper than hospital based teaching,(6) has some specific disadvantages, including geographical dispersal of tutors making quality control difficult,(7) and results in considerable travel costs to students.(8)
Most evaluations of community based teaching have concentrated on ascertaining student or faculty perceptions of its educational value.(9-11) Using an objective structured clinical examination, Satran et al found equal acquisition of clinical skills,(12) but in their study both community and hospital based students were taught by paediatricians and only two of the examination stations were patient based. There is little evidence on whether students taught in general practice can acquire their clinical skills as well as those taught in hospital. This study was designed to address this question.
Methods
The subjects were all first year clinical students at University College London Medical School during the academic year 1995-6. The school has a traditional medical curriculum with two years of basic science followed by three years of clinical medicine. After a four week introduction to clinical skills at the beginning of the first clinical year, students were randomly divided into four groups. Each group started with a different 10 week block (covering general medicine; surgery; medical specialties; and geriatrics, rheumatology, and orthopaedics) and rotated through all four blocks during the year. The block under study here (general medicine based at the Whittington Hospital) consisted of two five week attachments, one in general practice and one in hospital. Students were randomly allocated to start either in hospital or in the community and changed over after five weeks.
The intervention: hospital and general practice based
teaching
The five weeks in general practice - the "medicine in the
community" clerkship - at the Whittington was designed to replace a
traditional hospital clerkship and is described
elsewhere.(8) Both hospital and general practice attachments
shared common aims but the teaching structure was different (see box).
For example, as in most medical schools, the contracts of senior
academic and NHS staff are unspecific about teaching commitments,
whereas the general practitioners were paid specifically to provide
protected teaching time.
| Aims of the attachment | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Trial design and outcome measures
As well as determining whether students acquired their clinical
skills as well in the medicine in the community attachment as in the
hospital attachment we also wanted to see whether there was any
difference in the acquisition of specific skills in the two
environments.
The outcome measure was students' performance in two parallel nine station objective structured clinical examinations (P and Q, see box) given to all students at five and 10 weeks. In the first and third blocks students had P followed by Q, and in the second and fourth blocks they had Q followed by P. Two different examinations were used to promote test security, to minimise learning effects within the design, and to maximise the number of cases used, thus optimising the potential for detecting differential skill acquisition, overall, in the two environments. The disadvantage of using two examinations was that it was impossible to equate P and Q until after the first two blocks.
| Contents of examinations P and Q | ||
| Skill domain | P | Q |
| History taking | Acute severe breathlessness | Acute severe chest pain |
| Transient ischaemic attack | Chronic obstructive airways disease | |
| Rectal bleeding | Irritable bowel syndrome | |
| Physical examination | Respiratory system | Cardiovascular system |
| Peripheral nervous system: arm | Peripheral nervous system: leg | |
| Diabetic foot | Cranial nerves | |
| Communication skills | Hypertension | Asthma prophylaxis |
| Data interpretation | Ascites (photographs) | Rheumatoid arthritis and osteoarthritis (photographs) |
| Thyroid function test (hypothyroid patient) | Full blood count (microcytic, hypochromic anaemia) | |
Stations lasted 7 minutes and, except for two on data interpretation, each used a trained standardised patient. Stations were chosen to reflect problems found in both hospital medicine and general practice; all but two had been developed and validated over the previous five years. These two, on acute medical problems, were written by a consultant physician in consultation with a general practitioner. Examiners were drawn from senior hospital physicians and general practice tutors. Most were engaged in first year teaching; all others were experienced examiners in objective structured clinical examinations. Checklists, unavailable to students, were used for marking.
The performance of the students in hospital or community blocks
could be directly compared at the five week point for each of the four
blocks throughout the year; subsequently their improvement over the
s Analysis of data from the first two blocks suggested that
examination Q was slightly harder than P and that this was due to a
hard station in Q and an easy one in P. In particular, all students
scored so highly on the ascites station that there was no room for
improvement. Therefore the same two examinations were used for the
final two blocks, but with both photograph stations removed and the two
blood test stations used each time.
Results were analysed using SPSS 6.1 for Windows. Firstly, data
from each of the four blocks were analysed to compare the effect of the
two locations at the five week point. Total mean scores and mean scores
for each skill domain were compared using the t test for
unrelated groups. Subsequently improvements between 10 week and five
week scores were analysed using analysis of covariance, a statistically
equivalent variant of a design used by Ali et al(13) and
Nyquist et al(14) to compare two groups in a balanced
crossover design. In this analysis blocks 1 and 2 and blocks 3 and 4
were combined to use fully the balanced nature of the design. A total of 225 students rotated through the medicine in the
community firm in the study year. All took at least one objective
structured clinical examination, and 208 (92%) took both. Table 1
shows the results of a direct comparison of mean scores for the
hospital and community groups at five weeks for blocks 1 and 2. Table 2
displays the equivalent data for blocks 3 and 4. Table 3 shows the
incremental improvement in the two groups over the second five weeks of
the attachment for each of the four blocks. In Table 1 total
examination scores were higher for hospital students on examination P1
and for community students on examination Q2, but only the latter
reached statistical significance. These differences were due partly to
performance on the interpretation domain and to isolated differences on
individual stations, but not to other skill domains. There were no
consistent differences between locations. There were no such
differences for later groups, apart from one on data
interpretation.
Further analysis combined the first two and last two blocks in
separate anlayses. This showed that students' improvement over the
second five weeks, using the first examination as a covariate, was not
significantly different between learning locations for the first two
blocks (P=0.128) but was significantly better in the community than in
hospital for the last two blocks: mean improvement in score 6.6 (95%
confidence interval 3.4 to 9.8) for the community attachment and 1.7
(-1.3 to 4.7) for students studying in hospital (P=0.007). Further
analysis of scores broken down into skills showed that this was due to
improved examination skills in the community students (mean
improvement: community 4.23 (2.38 to 6.08), hospital 0.26 (-1.5 to
1.94; P<0.002)). These data represent a difference between locations
of about 3%-10% on five weeks' experience as measured by performance
in the examination.
In addition, mean scores for all students for each examination showed
gradual improvement over the 40 weeks. After equation of the
examinations, scores showed a significant monotonic linear trend
(P<0.001). The increase over the last 20 weeks was about 10%. In
other words, five weeks training in the community attachement, for
experienced first year clinical students, was worth about the
equivalent of 10 to 20 weeks of average clinical experience in terms of
performance in the examination.
The method was specifically chosen to sample as wide a range of skills
as possible within a balanced design. Its strengths include the
randomisation of students both to the order in which they took the four
blocks making up the first year and, once on the medicine block, to
starting in either community or hospital medicine. The crossover design
allowed both for direct comparisons of students taught in the two
venues and for pre- and post-exposure testing, a design which has been
widely favoured as a method of assessing efficacy of skills
teaching.(13-17) The comparison group is both plausible and
fair: the medicine in the community firm was expressly designed to
replace a traditional hospital medical clerkship and its brief was to
teach the same core clinical skills as those students learn in
hospital. The outcome measure, an objective structured clinical
examination, is a recognised and widely used method for testing
clinical skills(18) in undergraduate and postgraduate
settings.(19-20) The sample size in this study is large
enough to detect educationally important differences. Although longer
objective structured clinical examinations are needed before inferences
about a single student can be made, we were interested in group
performance, which is adequately determined by this number of stations.
Potential weaknesses in the design include the small number of
stations in each skill domain, which limits the power of the study to
determine comprehensively whether specific skills are acquired better
in either lo These data support current efforts to redistribute resources from
traditional locations for student learning to the community. It is
necessary to determine which specific knowledge, skills, and attitudes
are best acquired in the community, which are best acquired in
hospital, and which can be equally well acquired in either environment
given appropriate, well structured, and adequately resourced teaching.
Further work is also needed on which teaching methods optimise student
learning in all settings. Only then can we progress to rational
planning of new curricula and provide students with well structured
teaching and an optimum balance between hospital based and community
based learning.
We thank all the general practice tutors and consultant
physicians at the Whittington Hospital for their hard work as examiners
and Terri Charrier for organising the examinations.
Funding: Department of Health, through the Ce-MENT (community
based medical education in North Thames) project.
(Accepted 1 August 1997)
Department of Primary Care and
Population Sciences, Medical Education Unit,
Correspondence to: Dr
Murray
References
1 Whitehouse C, Roland M, Campion P.
Teaching medicine in the community: a guide for undergraduate
education. Oxford: OUP, 1997.
2 World Federation for Medical Education. Proceedings of the
World Summit on Medical Education. Med Educ
1994;28(suppl 1).
3 Robinson L A, Spencer J A, Jones R H. Contribution of academic
departments of general practice to undergraduate teaching and their
plans for curriculum development. Br J Gen Pract
1994;44:489-91.
4 Gordon P, Hadley J. Extending primary care:
polyclinics, resource centres, hospitals-at-home. London:
Radcliffe, 1996.
5 Audit Commission. What the doctor ordered. A study of
GP fundholders in England and Wales. London: HMSO,
1996.
6 Murray E, Jinks V, Modell M. Community-based medical
education: feasibility and cost. Med Educ 1995;29:66-71.
7 Price D A, Miflin B M, Mudge P R, Jackson C L. The quality of
medical teaching and learning in rurual settings: the learner's
perspective. Med Educ1994;28:239-51.
8 Murray E, Todd C, Modell M. Can general internal medicine be
taught in general practice? An evaluation of the University College
London model. Med Educ (in press).
9 Foldevi M, Trell E. Learning the basics of medicine in general
practice in the faculty of health sciences, Linkoping, Sweden.
Annals of Community-Oriented Education 1993;6:97-113.
10 Crebolder H F J M, Metsemakers J F M. General practice, medical
education and community-based teaching: the Maastricht experience.
Annals of Community-Oriented Education 1994;7:121-9.
11 Oswald N, Jones S, Date J, Hinds D. Long-term community-based
attachments: the Cambridge course. Med Educ
1995;29:72-6.
12 Satran L, Harris I B, Allen S, Anderson D C, Poland G A, Miller
W L. Hospital-based versus community-based clinical education: comparing
performances and course evaluations by students in their second-year
pediatrics rotation. Acad Med 1993;68:380-2.
13 Ali J, Cohen R, Reznick R. Demonstration of acquisition of
trauma management skills by senior medical students completing the ATLS
program. J Trauma 1995;38:687-91.
14 Nyqist JG, Naylor A J, Woodward-Lopez G, Dixon S. Use of
performance-based assessment to evaluate the impact of a skill-oriented
continuing education program. Acad Med 1994;10:S51-3.
15 Kokotailo P K, Langhough R, Neary E J, Matson S C, Fleming M F.
Improving pediatric residents' alcohol and other drug use clinical
skills: use of an experiental curriculum. Pediatrics
1995;96:99-104.
16 Berg D, Sebastian J, Heudebert G. Development, implementation,
and evaluation of an advanced physical diagnosis course for senior
medical students. Acad Med 1994;69:758-64.
17 Rothman A I. Understanding the objective structured clinical
examination. Aust NZ J Surg 1995;65:302-3.
18 Van der Vleuten C P M, Swanson D B. Assessment of clinical skills
with standardised patients: state of the art. Teaching and
Learning in Medicine 1990;2:58-76.
19 McFaul P B, Taylor D J, Howie P W. The assessment of clinical
competence in obstetrics and gynaecology in two medical schools by an
objective structured clinical examination. Br J Obstet
Gynaecol 1993;100:842-6.
20 Sloan D A, Donnelly M B, Schwartz R W, Strodel W E. The objective
structured clinical examination. The new gold standard for evaluating
postgraduate clinical performance. Ann Surg
1995;222:735-42.
21 Colliver J A, Barrows H S, Vu N V, Verhulst S J, Mast T A, Travis
T A. Test security in examinations that use standardised-patient cases
at one medical school. Acad Med 1991;66:279-82.
22 Rutala P J, Witzke D B, Leko E O, Fulginiti J V, Taylor P J. Sharing
of information by students in an objective structured clinical
examination. Arch Intern Med 1991;151:541-4.
23 Stillman P L, Haley H A, Sutnick A I, Philbin M M, Smith S R,
O'Donnell J, et al. Is test security an issue in a multistation
clinical assessment? A preliminary study. Acad Med
1991;66 (September suppl): S25-7.
24 Jolly B C, Cohen R, Newble D I, Rothman A. Possible effects of
reusing OSCE stations. Acad Med 1996;71:1023.
25 Niehaus A H, DaRosa D A, Markwell S J, Folse R. Is test security a
concern when OSCE stations are repeated across clerkship rotations.
Acad Med 1996;71:287-9.
Results
Table 1 - Comparison of mean scores on objective
structured clinical examinations (OSCE) of students taught for their
first five weeks of blocks 1 and 2 either in general practice or in
hospital Skill domain OSCE and
block Mean score
(95% CI)P
value General practice
Hospital History taking
(max score
60)P1 37.8
(36.2 to
39.4)39.9
(38.2 to 41.6)NS
Q2 38.4
(36.4 to
40.4)36.6
(34.3 to 39) NS Physical
examination
(max score 60)P1 35.9
(33.0 to
38.8)36.6
(33.3 to 40.0)NS
Q2 45.0
(41.8 to
48.2)41.
(39.9 to 43.5)NS
Communication skills
(max score
20)P1 12.0
(10.8 to
13.2)12.4
(11.7 to 13.2)NS
Q2 10.2
(9.4 to
11.0)10.1
(9.3 to 10.9)NS Data
interpretation (max score 40) P1 24.4
(23.0
to 25.8)26.9
(25.4 to 28.3)0.02
Q2 21.0
(19.4 to
22.6)17.4
(15.5 to 19.4)0.007 Total
score (max score 180) P1 110.1
(106.1 to
114.1)115.9
(111.7 to 120) NS
Q2 114.6
(109.2 to
120.1)105.9
(101.3 to
110.6)0.02 Students who took OSCE P1 (n=56) had had a four week introductory
course before their medical attachment; students who took OSCE Q2
(n=54) had had the four week introductory course plus 10 weeks of a
surgical attachment. Table 2 - Comparison of mean scores of students on
objective structured clinical examinations (OSCE) taught for their
first five weeks of blocks 3 and 4 in either general practice or in
hospital Skill domain OSCE and
block Mean score
(95%
CI)P value General
practice Hospital History taking
(max
score 60)P3 40.5
(38.4 to 42.6)39.9
(38.2 to 41.6)NS
Q4 41.1
(38.9 to
43.3)42.0
(40.0 to 44.0)NS
Physical examination
(max score
60)P3 42.9
(40.3 to
45.5)41.0
(38.9 to 43.1)NS
Q4 46.2
(44.3 to
48.1)46.4
(44.6 to 48.2)NS
Communication skills
(max score
20)P3 13.1
(12.3 to
3.9)14.0
(13.0 to 15.0)NS
Q4 13.4
(12.4 to
14.4)13.3
(12.2 to 14.4)NS Data
interpretation
(max score 40)P3 19.3
(17.5
to 21.1)22.0
(20.1 to 23.9)0.04
Q4 23.2
(21.8 to
24.6)24.0
((22.5 to 25.5)NS Total
score
(max score 180)P3 115.8
(110.0 to
121.6)116.9
(112.2 to 121.6)NS
Q4 123.9
(120.0 to
127.8)125.8
(121.8 to 129.8)NS Students who took OSCE P3 (n=57) had had 24 weeks of clinical
experience (4 weeks' introductory course, 10 weeks' surgery, and 10
weeks' medical specialties) while students who took OSCE Q4 (n=55)
were on the last block of the year. Table 3 - Incremental improvement in total scores on
the objective structured clinical examination (OSCE) for the two groups
over the second five weeks of their attachments Block Basis of
improvement Mean improvement
(95%
CI)P value General
practice Hospital
1 Q1 - P1 (t
test) 5.4
(0.12 to 10.8)2.0
(-3.0 to 7.0)NS 2 P2 - Q2 (t
test) 25.1
(18.2 to 32) 29.3
(22.6 to 36) NS 1 and 2 Balanced analysis of
covariance 17.35
(12.35 to 22.35)13.68
(9.04 to 18.32)NS 3 Q3*-P3* (t
test) 7.4
(2.4 to 12.4) 3.5
(-0.7 to 7.7)NS 4 P4*-Q4* (t
test) 5.5
(1.9 to 9.2)-.33
(-4.4
to 3.8) 0.041 3 and 4 Balanced analysis of
covariance 6.61
(3.4 to 9.8) 1.7
(-1.3 to 4.7)0.007 *OSCEs P and Q modified as described in the text Discussion
The results suggest that, overall, students acquire their clinical
skills as well, if not better, in general practice as in hospital. This
appears to be true for all the skill domains tested. In particular,
examination skills improved more for experienced students in community
locations. Generally students' clinical skills continued to improve
throughout the year. The spread of results around the mean diminished
throughout the year, so it is possible that there is a ceiling effect:
given the sort of experience provided in the first clinical year,
students can only progress so far.
Key Messages
Students can learn clinical method as well in general practice
as in hospital This supports efforts to redistribute resources from
traditional learning locations to general practice More work is needed to determine which specific skills,
knowledge, and attitudes are best acquired in each location.
Conflict of interest: None.
University College London Medical School and Royal
Free Hospital School of Medicine,
Whittington Hospital,
London N19
5NF
Elizabeth Murray, senior lecturer in
primary health care
Michael
Modell, professor of primary health
care
University of Leeds,
Leeds
LS2 9NL
Brian Jolly,
director
Home | Current contents | Past issues |
Classified ads | Career Focus | Feedback
Collections |
About this site | About the BMJ | BMA | Medline