Cohort study of examination performance of undergraduate medical students learning in community settingsBMJ 2004; 328 doi: http://dx.doi.org/10.1136/bmj.328.7433.207 (Published 22 January 2004) Cite this as: BMJ 2004;328:207
- 1 Flinders University Rural Clinical School, Flinders University, GPO Box 2100, Adelaide, SA, 5001, Australia
- 2 Department of General Practice, Flinders University
- 3 Department of Medical Education, Flinders University
- Correspondence to: P Worley
- Accepted 10 October 2003
Objectives To determine whether moving clinical medical education out of the tertiary hospital into a community setting compromises academic standards.
Design Cohort study.
Setting Flinders University four year graduate entry medical course. In their third year, students are able to choose to study at the tertiary teaching hospital in Adelaide, in rural general practices, or at Royal Darwin Hospital, a regional secondary referral hospital.
Participants All 371 medical students who did their year 3 study from 1998-2002.
Main outcome measures Mean student examination score (%) at the end of year 3.
Results The unadjusted mean year 3 scores at each location differed significantly (P < 0.001); the mean score was 65.2 (SE = 0.43) for Adelaide students, 68.2 (0.83) for Darwin students, and 69.3 (0.97) for students on the rural programme. Mean year 2 scores were similar for each location. Post hoc tests of means adjusted for sex, age, year 2 score, and cohort year showed that the rural and Darwin groups had a significantly improved score in year 3 compared with the Adelaide group (adjusted mean difference = 3.08, 95% confidence interval 1.25 to 4.90, P < 0.001 for rural group; 1.91, 0.47 to 3.36, P = 0.001 for Darwin group).
Conclusions These findings show that the concern that student academic performance in the tertiary hospital would be better than that of students in the regional hospital and community settings is not justified. This challenges the orthodoxy of a tertiary hospital education being the gold standard for undergraduate medical students.
Undergraduate medical education is increasingly being transferred out of tertiary hospitals into community settings despite scant evidence of the effect this has on academic performance.1–3 A Medline search with the key words “undergraduate, medical education, tertiary, community, comparison” produced only one reference, a description of the programme on which this paper is based. This lack of evidence leaves community based medical education open to the attack of being a fad, driven by idealists who are prepared to compromise academic standards for their social agendas.4 In this study, we examined whether academic standards at our institution were compromised by moving medical education out of the tertiary hospital.
Flinders University delivers a four year graduate entry medical curriculum using problem based learning as the main method of instruction.5 The first two years of the course are predominantly taught in small group tutorials and the last two years almost entirely in clinical settings. The main clinical examination occurs at the end of the third year.
During year 3, all students study medicine, surgery, paediatrics, women's health, general practice, and liaison psychiatry. At the 500 bed urban university tertiary teaching hospital in Adelaide, the students learn each discipline in sequential rotations, with the exception of liaison psychiatry, which is integrated throughout the year.
In 1997, the school piloted an innovative community based option for year 3 students, the parallel rural community curriculum.6 The curriculum enables up to eight students a year to do their entire year 3 studies based in small (four to eight doctors) rural general practices in the Riverland region of South Australia, 250 km from Adelaide. In 2002, a further eight places were made available in the Greater Green Triangle region of south eastern South Australia and western Victoria. This programme was developed with the help of the Commonwealth Department of Health and Aging as part of a long term strategy to tackle the shortage of doctors in rural Australia.
In 1998, the school started teaching the year 3 curriculum at Royal Darwin Hospital, a 200 bed regional secondary referral hospital 3000 km from Adelaide in the tropical north of the country. This hospital had previously managed short term elective teaching for medical students from several medical schools but had not been seen as a teaching hospital or had academic staff. Up to 16 students a year do all their year 3 study in Darwin. This opportunity has been made possible by the support of the Northern Territory government.
The separate governmental assistance to the two programmes has enabled them to develop without having to rely on funds from the tertiary centre. Most of the funding, however, has come from money allocated to pay for teaching for local general practitioners and specialists. In the tertiary centre, the course is usually funded through teaching hospital funds and university operating grants. As the number of students at the tertiary centre is now smaller, both the rural communities and the tertiary hospital have gained financially. The level of resources for each student is similar for all of the programmes despite the different sources of funds.
All students (70-90 a year) did their first two years of medical study at Flinders Medical Centre. Students were then able to apply to the Darwin or rural community programmes. As there were usually more applicants than places, we selected students on the basis of an autobiographical statement and interview, with an emphasis on their intention for future practice in a rural or remote setting. Academic results for year 2 were not available at the time of selection.
At the end of year 3, all students sat an identical clinical examination consisting of written multiple choice questions, extended matching questions, modified essay questions, mini-case written papers, and a comprehensive objective structured clinical examination.
We included all students who did their year 3 study in 1998-2002. The main outcome measure was examination score (%) in year 3. We tested differences between locations by sex, age, and cohort year using exact χ2 tests and differences by examination score in year 2 and year 3 using one way analysis of variance. We used analysis of covariance to compare the mean score in year 3 between locations after adjusting for covariates. Covariates included score in year 2, sex, age, and cohort year. The cohort year was first converted to four dummy variables, each representing a comparison between a given year and 1998. We also did a Dunn-Sidak post hoc comparison of adjusted means. We used the Stata 8.0 package for statistical analyses.
There were 371 students included in the study. Table 1 shows the sex, age, and cohort year by location. None of these variables differed significantly by location. Mean score for year 2 scores were similar for each location, but mean scores for year 3 differed significantly (table 2).
Table 3 shows the results of the analysis of covariance. The model accounted for 61% of the variance in year 3 score. In particular, there was a highly significant difference in year 3 score by location after we had adjusted for covariates.
The Dunn-Sidak post hoc tests found that the rural programme group had a significantly improved score in year 3 over the Adelaide group (adjusted mean difference = 3.08, 95% confidence interval 1.25 to 4.90; P < 0.001). The Darwin group also had a significantly better score in year 3 than the Adelaide group (adjusted mean difference = 1.91, 95% confidence interval 0.47 to 3.36; P = 0.001).
We know of no other studies directly comparing the academic performance of students learning the same curriculum in tertiary, secondary, or primary care settings. Examination performance is only a proxy for academic performance, but it is the most common measure used by universities to determine academic progression, is used by many hospitals in ranking applicants for junior doctor positions, and is a measure that students value highly.
We did not use randomization in this study. This is a common problem in educational settings, introducing the possibility of biases, including selection bias.7 It could be argued that the students who were selected for the rural and remote settings were, for example, more resourceful and talented. If this were so, it would have been evident in their previous academic performance because the first two years' study, based entirely on problem based learning, encourages and rewards these qualities. However, we saw no differences in previous academic performance by group (table 2).
A further argument against selection bias being a serious limitation in this study is the school's selection procedure. The three elements that contribute to this process are performance in a national admission test of reasoning and problem solving in the basic sciences and humanities, performance in a previous undergraduate degree, and performance at a standardised interview that rewards evidence of self directedness, teamwork, communication skills, compassion, resourcefulness, and broad life skills. Thus, all medical students at Flinders are likely to be resourceful and talented high achievers.
Other qualities that could be over-represented in students in the rural programme and Darwin groups, such as seeking adventure, interest in rural medicine as a career, being suited to rural life, and having fewer ties with the city, show only the student centred benefits of offering different environments to suit the needs of different students. There is no suggestion that such qualities inherently affect academic performance, but when a student's self perceived traits are matched to an environment that supports them, their academic performance may be enhanced. This should, however, have been equally true for students who chose to study in the tertiary centre.
Our analysis accounted for the other possible biases of student age, sex, year 2 score, and cohort effect. The results cannot be explained by the Hawthorne effect as they have been sustained over five years and during this time the rural and Darwin programmes have become a routine part of the school's study options. The results may also be criticised for reflecting the effect of group size rather than location. However, on a day to day basis, students in all of the locations were allocated to practices or wards in pairs or alone, and tutorial group sizes were also similar.
Future of community learning
When the rural and Darwin programmes were initiated, the university had some concerns that student learning would be compromised in pursuit of the longer term workforce aims. The quality of the students' examination performance in the regional hospital, and, in particular, in the community setting, has allayed this concern. Our findings challenge the orthodoxy of a tertiary hospital being the most appropriate location for all undergraduate medical students.
In 2001, the Australian Commonwealth government announced a national programme that will require each medical school to enable a quarter of its students to undertake half of their clinical education in rural or remote settings. Although some students will learn better in large urban settings, our findings should give students confidence that they do not have to sacrifice academic performance when taking advantage of such learning opportunities.
Contributors PW conceived and led the research and manuscript preparation. AE led the statistical analysis of the data. DP contributed to the study design and to the analysis of the educational implications of the findings. PW is the guarantor.
Funding The parallel rural community curriculum is funded through a grant from the Commonwealth Department of Health and Aged Care. The Royal Darwin Hospital programme is principally funded by the Northern Territory Government.
Competing interests None declared.