BMJ 2001;322:40-44 ( 6 January )

Education and debate

Computer assisted learning in undergraduate medical education

Trisha Greenhalgh, senior lecturer in primary care. 

Open Learning Unit, Department of Primary Care and Population Sciences, Royal Free and University College Medical School, London N19 3UA

p.greenhalgh{at}ucl.ac.uk

It is becoming "a truth universally acknowledged" that the education of undergraduate medical students will be enhanced through the use of computer assisted learning. Access to the wide range of online options illustrated in the figure must surely make learning more exciting, effective, and likely to be retained. This assumption is potentially but by no means inevitably correct.


Box 1 : Why fund computer assisted learning?

Computer assisted learning is inevitable---Individual lecturers and departments are already beginning to introduce a wide range of computer based applications, sometimes in a haphazard way. Planned and coordinated development is better than indiscriminate expansion
It is convenient and flexible---Courses supported by computer assisted learning applications may require fewer face to face lectures and seminars and place fewer geographical and temporal constraints on staff and students. Students at peripheral hospitals or primary care centres may benefit in particular
Unique presentational benefits---Computer presentation is particularly suited to subjects that are visually intensive, detail oriented, and difficult to conceptualise, such as complex biochemical processes or microscopic images.1 Furthermore, "virtual" cases may reduce the need to use animal or human tissue in learning
Personalised learning---Each learner can progress at his or her preferred pace. They can repeat, interrupt, and resume at will, which may have particular advantages for weaker students
Economies of scale---Once an application has been set up, the incremental cost of offering it to additional students is relatively small
Competitive advantage---Potential applicants may use the quality of information technology to discriminate between medical schools. A "leading edge" virtual campus is likely to attract good students
Achieves the ultimate goal of higher education---The goal is to link people into learning communities. Computer applications, especially the internet and world wide web, are an extremely efficient way of doing this2
Expands pedagogical horizons---The most controversial argument for using computer assisted learning in higher education is the alleged ability of the virtual campus to alter fundamentally the relation between people and knowledge3

Deans of medical faculties often receive requests for development funding for computer assisted learning projects. Decisions to introduce these projects into the undergraduate curriculum are generally justified by one or more of the arguments listed in box 1.


Summary points

Reduced funding, rising student numbers, geographical dispersal, and increased competition in a complex global market have put medical schools under pressure to embrace computer assisted learning
New technologies may have important educational advantages, but without support and training for staff and students they could prove an expensive disaster
Expansion of computer assisted learning requires cultural change as well as careful strategic planning, resource sharing, staff incentives, active promotion of multidisciplinary working, and effective quality control




    Developing applications

Computer assisted learning applications generally require the student to follow the content without immediate or direct supervision from the tutor. But the computer can be a temperamental and unforgiving beast, and computer assisted learning applications must therefore embody the quality features described in box 2. For all these reasons, computer assisted learning materials are initially much more labour intensive and time consuming to prepare than most face to face courses, and they often require input from fairly senior members of staff. Once the basic format is agreed and the initial materials have been written, however, materials can be maintained and updated relatively easily and by more junior members. Off the shelf templates that allow someone with no specific training to produce materials of professional quality are increasingly available. Introducing computer assisted learning technologies into a traditional course will generally occur in stages, as described in box 3. Adapting pre-existing materials designed as handouts or revision notes can sometimes save considerable time.


    Fulfilling its potential

Educationists are excited about the potential of so called third generation distance education technologies to provide a "rich environment for active learning"5 in which the learner actively builds rather than passively consumes knowledge. This requires a transformed view of the nature of knowledge itself as dynamic, open ended, multidimensional, and public rather than static, finite, linear, and private.

Computer technologies can support a wide range of learning activities which engage students in a continuous collaborative process of building and reshaping understanding. Yet despite theoretical appeal and broadly positive results from a handful of randomised trials conducted by enthusiasts (table), the real advantages of computer assisted learning in medical curricula outside the research setting have yet to be shown consistently.


    Published studies

Few articles on computer assisted learning in medical education have been published. A search of Medline and ERIC databases using the Mesh term "medical education" and free text terms "computer based" and "computer assisted" turned up around 200 potentially relevant studies, of which only 12 were prospective randomised studies with objective, predefined outcome criteria (table). These studies represent a range of different settings, interventions, and outcomes and are therefore not directly comparable. Most studies have methodological problems, including lack of statistical power, potential contamination between intervention and control groups, and attrition of the sample.


Box 2 : Quality features of applications

Open learning (self study) materials---Applications must be prepared in advance, cover explicit course aims and learning objectives, and include a high degree of "signposting," explanatory text, and trouble shooting information
Website design---Websites should have a logical structure and sequence, utilise features such as hypertext and graphics, and include links to public access, web based materials such as electronic journals where appropriate
Technical design---Applications must be user friendly and operate effectively within the hardware and software constraints of the end user
University culture---Applications must present an academic ethos in the untidy, commercial, and laissez-faire culture of web based publishing and protect the student from the distractions of the internet
Copyright protection---Applications should reflect an agreed university policy on intellectual property rights


Box 3 : Stages in integrating web technology into a traditional degree course (adapted from Devitt and Palmer4)

Level 1---Course has a public access web page directed at potential students
Level 2---Course materials include links to public access websites
Level 3---Students generate web based resources (glossaries, topic summaries, etc) and share them on an internal website
Level 4---Key course resources (lecture notes, slides, etc) are available on an internal website
Level 5---Students prepare materials based on course content (for example, clinical case histories) for other learners and publish them on the internet
Level 6---Participation in web based learning activities is a course requirement (for example, a course based bulletin board)
Level 7---Participation in web based learning activities extending beyond the class is a course requirement (for example, membership of an academic list server)
Level 8---Web serves as an alternative delivery mechanism for resident students (range of interactive materials available for asynchronous access to students on campus, who may choose to work with these rather than attend some face to face classes)
Level 9---Entire course is available on the web for students to access anywhere (includes full electronic syllabus, teleconferencing, facilities for sending and receiving attachments, and collaborative coursework)
Level 10---Fully web based course is part of larger programmatic web initiative (for example, students may collect transferable credits from this course towards a trans-university degree)



View larger version (39K):
[in this window]
[in a new window]
  		Types of computer assisted learning materials available

As the table shows, the randomised controlled trials had mixed but generally positive results. These suggested that the efficacy (the "can it work?" question described by Haynes17) of high quality programmes in medical education is reasonably well established, a finding that is in keeping with meta-analyses of computer assisted learning in non-medical education.18 However, the effectiveness and cost effectiveness of these initiatives remain in doubt.

In the mid-1990s, at least two UK medical schools supplied all first year students with laptop computers and enhanced access to a range of networked multimedia applications. One project was never formally evaluated, but anecdotal reports suggested that many students found the computers expensive, impractical, and difficult to integrate with the mainstream curriculum (P Booton, personal communication). Results of the other project were published. The authors bravely admitted that some students made no use of their computers at all, technical glitches and incompatibility problems were common, staff were ill prepared for the change in learning medium, and "there was no academic organisational structure to shape a coherent response to the rapid increase in computer use."19


                              
View this table:
[in this window]
[in a new window]
 

Published randomised controlled trials of computer assisted learning (CAL) methods in undergraduate medical education

Lack of engagement
Failure of students to engage with newly introduced technology is a recurring theme in reports on non-medical education. Perceived barriers include inadequate planning, poor integration with other forms of learning, and cultural resistance from staff. One ethnographic study in which students were closely observed while taking part in online courses showed that considerable frustration and time wasting arose from poor course design, technical glitches, "dead" hypertext links, poorly coordinated real time seminars, and ambiguous instructions.20 The only study of computer assisted learning in medical education that used comparable, in depth, qualitative methods found few such problems, but it was restricted to students' use of computers in a supervised classroom setting.21

Transferability and evaluation
Three important conclusions can be drawn from the reports. Firstly, innovators who have developed apparently successful products should be guarded about claiming that their systems are transferable, even when the efficacy of these systems has been shown in the research setting. Secondly, the evaluation of all educational technologies should include observation of unsupervised students attempting to gain access from remote sites and follow online links and instructions. Thirdly, neither course materials nor teaching skills are directly transferable from the traditional lecture theatre to the virtual campus. We should recognise, and take systematic steps to guard against, the danger of allowing inadequately trained tutors and lecturers to "go virtual."



View larger version (66K):
[in this window]
[in a new window]
  		Example of computer assisted learning application used at UCL Medical School

Learning culture
The differences in learning culture between computer based and traditional learning should not be underestimated, especially for the novice. As Reingold argues, "Fear is an important element in every novice computer user's first attempts to use a new machine or new software: fear of destroying data, fear of hurting the machine, fear of seeming stupid in comparison to others, or even to the machine itself."22

One author has distinguished between students who "lose themselves" and those who "find themselves" in the virtual environment of email discussions,23 and another found that whereas some students perceived their virtual seminar group as part of a warm, friendly, and supportive online community, others perceived themselves facing a whole sea of strangers, perhaps reflecting different stages in the development of online learning skills (box 4) or different learning styles.24


Box 4 : Stages of competence in online learning (adapted from Salmon24)

Level 1---Gaining access. Is able to log on and motivated to continue; posts first "joining" message when instructed
Level 2---Becoming familiar with the online environment. Possesses basic technical skills and is confident in sending and receiving messages to and from tutor and other students
Level 3---Seeking and giving information. Is confident in using all features of the software. Freely offers, receives, and processes information from others on line
Level 4---Knowledge construction. Shows actions on line that are likely to lead to knowledge construction, including creative and active thinking (asking challenging questions, reflecting, suggesting ideas) and interactive thinking (critiquing, negotiating interpretations, summarising, proposing actions based on ideas)
Level 5---Autonomy and development. Takes responsibility for own continuing development in online learning. Is able to set up and support own virtual group

Issues of costs and training
The cost of hardware and software, and telephone line charges, often prove a more important barrier to accessing web based materials than the course organisers initially assume. The amount of training needed to become comfortable with specialised software packages is often underestimated; students on a course that relies heavily on computer work may spend most of their first term getting to grips with the technology. Few students learn all the essential technical skills at the outset of the course. Rather, they tend to use "just in time learning"---that is, most of them make no attempt to get to grips with a feature of the software until they actually need to use that feature. This suggests that too much initial training may not be popular or effective


    Recommendations for introducing computer assisted learning

Invest in staff development
Developing computer assisted learning applications is a lengthy and skilled process. Innovators within traditional courses have embraced the concept and have often produced creative and high quality material to supplement their existing courses. But these individuals are in a minority; most academics will not become developers or supporters of computer assisted learning unless considerable time and resources are dedicated to supporting this activity.25 Staff who are sent on "generic" workshops designed to improve their use of computer assisted learning technologies may complain afterwards that they still do not know where to start and feel that the time was not well spent.26 For all these reasons, staff training should be tightly targeted and be offered on a project by project basis.

Provide a central resource base
Avoid reinventing the wheel. Templates, models, and images developed for one course may also serve another course within the same institution (and even beyond it). Mechanisms to allow exchange of skills, resources, and ideas between institutions must be put in place early, as exemplified by the University of Aberdeen Medical School's structured approach to the development of computer assisted learning.27 In addition, medical schools must identify and become part of wider networks that are already sharing and working collaboratively on materials, such as the UK Assisting Collaborative Education Project.28

Aim to use different methods
Academics generally construct courses in a somewhat haphazard way from prepared lectures, handouts, photocopies of book chapters, reading lists, journal articles, laboratory notes, case studies, and so on. Hence, the vision of a degree course that is completely virtual---high tech, fully integrated, stand alone, based entirely on computer applications, and difficult to upgrade---is unlikely to become the model for the typical course of the future. Rather, computer assisted learning products are most likely to be used by academics if they are easily customised, capable of being modified, upgraded, and integrated with traditional teaching material, and discarded as soon as their useful life is past.

Staff incentives
Intensive and continuing central support for departmental initiatives should be linked to appropriate incentives and rewards for individual staff who become active members of the virtual campus. These should be both internal (for example, included in criteria for promotion) and external (for example, accreditation via the Institute of Learning and Teaching or the Association for Learning Technologies).

Multidisciplinary working
The development of computer based teaching and learning materials requires expertise in content, in pedagogy, and in technical aspects of design and delivery. Staff with most to offer in the way of technical design may overlook important educational principles, and those who focus on content may make incorrect assumptions about the ability of the technology to deliver their imaginative ideas. A multidisciplinary, team based approach is likely to be the most successful model for working.

Address issues of organisational culture
Introducing interactive learning technology is a contemporary case study of the difficulties involved in embedding new ideas and new ways of working into institutions that are resistant to change. Lessons can be drawn from strategic change theory; essential steps include creating readiness for change, energising commitment, developing political support, managing the transition, and sustaining momentum. Resistance to change is most likely to come from the underlying culture of the organisation---that is, values, ways of thinking, management styles, and pedagogical paradigms.29

Initiatives to develop computer based materials usually begin as distinct projects with management and development processes separate from, and parallel to, the existing structures and processes of the university. This inevitably limits the impact of the initiative in terms of its benefits to the whole organisation, increases costs through duplication, and imposes limits to its expansion and continuation. Ad hoc innovations in computer assisted learning, whether conceived of as pilot or pump priming projects, frequently fade away when the individuals associated with them move on to other activities.30

Developing a university from a position in which it supports small, discrete, department based initiatives to one in which the virtual campus is embedded in established mainstream activities is complex. Stress lines may appear in a number of areas, notably student administration, student support, quality assurance, staff development policies and priorities, infrastructure development, financial management, and resource priorities.30


    The future

Many medical schools are discovering the prohibitive cost of producing high quality computer assisted learning materials. In the spirit of Dr Blunkett's collaborative e-university, a new form of academic commerce in off the shelf, web based course materials is beginning to emerge.31 Agreements between universities (and countries) on sharing units of education may eventually lead to the awarding of a degree that cannot be identified with a single institution.25 Funding of a medical degree may even begin to occur on a module by module basis and, arguably, is less likely to come from a single central source.32 The medical school of the future may be one that can successfully offer (in collaboration with other educational providers) a flexible menu of both face to face and self study modules from which individual students can select to meet their own unique requirements. Any other option, including staying as we are, may ultimately prove unaffordable.

    Acknowledgments

I am grateful to Professor Lewis Elton for helpful comments on an earlier draft of this paper. The views expressed are mine alone.

    Footnotes

Competing interests: None declared.


    References

1. Phillips R. Developers' guide to interactive multimedia. A methodology for educational applications. Perth: Curtin University Press, 1996.
2. Haag M, Maylein L, Leven FJ, Tonshoff B, Haux R. Web-based training: a new paradigm in computer-assisted instruction in medicine. Int J Med Informatics 1999; 53: 79-90.
3. Daniel JS. The knowledge media. In: Mega-universities and knowledge media. Technology strategies for higher education. London: Kogan Page, 1996:101-135.
4. Devitt P, Palmer E. Computer-aided learning: an overvalued educational resource? Med Ed 1999; 33: 136-139.
5. Grabinger S, Dunlap JC. Rich environments for active learning: a definition. www.warwick.ac.uk/alt-E/rolling/123 (accessed 30 Oct 2000).
6. Thorne M, ed. Universities in the future. London: Department of Trade and Industry, 1999.
7. Carr MM, Reznick RK, Brown DH. Comparison of computer-assisted instruction and seminar instruction to acquire psychomotor and cognitive knowledge of epistaxis management. Otolaryngol Head Neck Surg 1999; 121: 430-434[CrossRef][Medline].
8. D'Alessandro DM, Kreiter CD, Erkonen WE, Winter RJ, Knapp HR. Longitudinal follow-up comparison of educational interventions: multimedia textbook, traditional lecture, and printed textbook. Acad Radiol 1997; 4: 719-723[CrossRef][Medline].
9. Elves AW, Ahmed M, Abrams P. Computer-assisted learning; experience at the Bristol Urological Institute in the teaching of urology. Br J Urol 1997; 80 (suppl 3): 59-62.
10. Hilger AE, Hamrick HJ, Denny Jr FW. Computer instruction in learning concepts of streptococcal pharyngitis. Arch Pediatr Adolesc Med 1996; 150: 629-631[Abstract].
11. Kallinowski F, Mehrabi A, Gluckstein C, Benner A, Lindinger M, Hashemi B, et al. Computer-based training---a new method in surgical education and continuing education. Chirurg 1997; 68: 433-438[CrossRef][Medline].
12. Lyon Jr HC, Healy JC, Bell JR, O' Donnell JF, Shultz EK, Wigton RS, et al. Significant efficiency findings while controlling for the frequent confounders of CAI research in the PlanAlyzer project's computer-based, self-paced, case-based programs in anemia and chest pain diagnosis. J Med Systems 1991; 15: 117-132[CrossRef][Medline].
13. Mehta MP, Sinha P, Kanwar K, Inman A, Albanese M, Fahl W. Evaluation of internet-based oncologic teaching for medical students. J Cancer Educ 1998; 13: 197-202[Medline].
14. Rogers DA, Regehr G, Yeh KA, Howdieshell TR. Computer-assisted learning versus a lecture and feedback seminar for teaching a basic surgical technical skill. Am J Surg 1998; 175: 508-510[CrossRef][Medline].
15. Schwartz S, Griffin T. Comparing different types of performance feedback and computer-based instruction in teaching medical students how to diagnose acute abdominal pain. Acad Med 1993; 68: 862-864[Medline].
16. Summers AN, Rinehart GC, Simpson D, Redlich PN. Acquisition of surgical skills: a randomized trial of didactic, videotape, and computer-based training. Surgery 1999; 126: 330-336[Medline].
17. Weverling GJ, Stam J, ten Cate TJ, van Crevel H. Computer-assisted education in problem-solving in neurology; a randomized educational study. Nederlands Tijdschrift voor Geneeskunde 1996; 140: 440-443[Medline].
18. Haynes B. Can it work? Does it work? Is it worth it? The testing of healthcare interventions is evolving. BMJ 1999; 319: 652-653[Free Full Text].
19. Kulik JA. Meta-analytic studies of findings on computer-based instruction. In: Baker EL, O'Neil HF, eds. Technology assessment in education and training. Hillsdale, NJ: Erlbaum, 1994:52-66.
20. Conroy RM, Brazier H, Clarke E. Giving laptop computers to medical students. CTITM Update 1999; 9: 10-11.
21. Hara N, Kling R. Students' frustrations with a web-based distance education course: a taboo topic in the discourse. www.slis.indiana.edu/CSI/wp99_01.html (accessed 30 Oct 2000).
22. Lehmann HP, Freedman JA, Massad J, Dintzis RZ. An ethnographic, controlled study of the use of a computer-based histology atlas during a laboratory course. J Am Med Informatics Assoc 1999; 6: 38-52[Abstract/Free Full Text].
23. Reingold H. The virtual community. London: Minerva, 1995:10.
24. Salmon G. E-moderating: a guide to tutoring and mentoring on line. London: Kogan Page, 2000.
25. Bonk CJ, Cummings JA, Hara N, Fischler RB, Lee SM. A ten level web integration continuum for higher education: new resources, partners, courses, and markets. In: Abbey B, ed. Instructional and cognitive impacts of web-based education. University of Indiana, 2000. (in press). php.indiana.edu/~cjbonk (accessed 30 Oct 2000).
26. Daniel JS. The technology adoption life-cycle. In: Mega-universities and knowledge media. Technology strategies for higher education. London: Kogan Page, 1996:88-90.
27. Conole G. Integration through iteration: the project-based approach. Proceedings of the learning technology life cycle 6th international conference, Bristol, 21-23 September 1999. www.ilrt.bris.ac.uk/alt-c99/ips-day1.htm (accessed 16 Nov 2000).
28. Hamilton NM, Furnace J, Duguid KP, Helms PJ, Simpson JG. Development and integration of CAL: a case study in medicine. Med Ed 1999; 33: 298-305.
29. Assisting Collaborative Network. ace.ac.uk (accessed 30 Oct 2000).
30. Elton L. New ways of learning in higher education: managing the change. Tertiary Educ Manage 1999; 5: 207-225.
31. Brown S. Reinventing the university. Assoc Learning Technol J 1999; 6: 30-37.
32. Fender B. The e-university project. London: Higher Education Funding Council for England, April 2000. (Circular letter to vice chancellors and principals 04/00.) www.hefce.ac.uk/pubs/Circlets/2000/cl04_00.htm (accessed 30 Oct 2000).

(Accepted 10 August 2000)

© BMJ 2001
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?

Related Article

Computer assisted learning aids management of course information
Paul Whatling, W Angus Wallace, and Andrew Lamb
BMJ 2001 322: 1428. [Extract] [Full Text]

This article has been cited by other articles:

  • Ashton, A, Bhati, R (2007). The use of an asynchronous learning network for senior house officers in emergency medicine. Emerg. Med. J. 24: 427-428 [Abstract] [Full text]  
  • Morton, D. A., Foreman, K. B., Goede, P. A., Bezzant, J. L., Albertine, K. H. (2007). TK3 eBook software to author, distribute, and use electronic course content for medical education. Adv. Physiol. Educ. 31: 55-61 [Abstract] [Full text]  
  • Hare, E. H., Evans, P., McIntosh, C. E., Thomson, L. D. G. (2007). Case-based online learning for medical undergraduates. Psychiatr. Bull. 31: 73-75 [Full text]  
  • Bradley, P., Oterholt, C., Nordheim, L., Bjorndal, A. (2005). Medical Students' and Tutors' Experiences of Directed and Self-Directed Learning Programs in Evidence-Based Medicine: A Qualitative Evaluation Accompanying a Randomized Controlled Trial. Eval Rev 29: 149-177 [Abstract]  
  • Cottrell, D. (2003). The APT online discussion forum. Adv. Psychiatr. Treat. 9: 84-85 [Full text]  
  • Greenhalgh, T., Toon, P., Russell, J., Wong, G., Plumb, L., Macfarlane, F. (2003). Transferability of principles of evidence based medicine to improve educational quality: systematic review and case study of an online course in primary health care. BMJ 326: 142-145 [Full text]  
  • Harris, J. M Jr, Salasche, S. J, Harris, R. B (2001). Infopoints: The internet and the globalisation of medical education. BMJ 323: 1106-1106 [Full text]  
  • Wallace, G. (2001). Information technology and telemedicine. CMAJ 165: 777-779 [Full text]  
  • Whatling, P., Wallace, W A., Lamb, A. (2001). Computer assisted learning aids management of course information. BMJ 322: 1428-1428 [Full text]  

Rapid Responses:

Read all Rapid Responses

Long established internet course
Peter Hall
bmj.com, 9 Jan 2001 [Full text]
Computer-assisted learning in postgraduate education
David Lewis
bmj.com, 9 Jan 2001 [Full text]
EasyCAL, Computer Assisted Learning need not be painful in the 21st Century
Richard Cheong-Leen
bmj.com, 11 Jan 2001 [Full text]
CAL enhances course information management
Paul Whatling
bmj.com, 15 Jan 2001 [Full text]

Student BMJ

Sepsis

The latest guidlines will affect how we practice medicine

www.student.bmj.com

Listen to the latest BMJ Interview