- Christian R Larsen, clinical research fellow1,
- Jette L Soerensen, assistant professor and consultant2,
- Teodor P Grantcharov, assistant professor and consultant3,
- Torur Dalsgaard, consultant4,
- Lars Schouenborg, consultant4,
- Christian Ottosen, consultant4,
- Torben V Schroeder, professor and consultant5,
- Bent S Ottesen, managing director and professor at the Juliane Marie Centre6
- 1Department of Gynecology, Juliane Marie Centre for Children, Women and Reproduction, Copenhagen University Hospital Rigshospitalet, Blegdamsvej 9, DK-2100 OE, Copenhagen, Denmark
- 2Department of Obstetrics, Juliane Marie Centre, Copenhagen
- 3Division of General Surgery, St Michael’s Hospital, Toronto, ON, Canada
- 4Department of Gynecology, Juliane Marie Centre, Copenhagen
- 5Department of Vascular Surgery, Abdominal Centre, Copenhagen University Hospital Rigshospitalet, Copenhagen
- 6Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
- Correspondence to: C R Larsen
- Accepted 21 January 2009
Objective To assess the effect of virtual reality training on an actual laparoscopic operation.
Design Prospective randomised controlled and blinded trial.
Setting Seven gynaecological departments in the Zeeland region of Denmark.
Participants 24 first and second year registrars specialising in gynaecology and obstetrics.
Interventions Proficiency based virtual reality simulator training in laparoscopic salpingectomy and standard clinical education (controls).
Main outcome measure The main outcome measure was technical performance assessed by two independent observers blinded to trainee and training status using a previously validated general and task specific rating scale. The secondary outcome measure was operation time in minutes.
Results The simulator trained group (n=11) reached a median total score of 33 points (interquartile range 32-36 points), equivalent to the experience gained after 20-50 laparoscopic procedures, whereas the control group (n=10) reached a median total score of 23 (22-27) points, equivalent to the experience gained from fewer than five procedures (P<0.001). The median total operation time in the simulator trained group was 12 minutes (interquartile range 10-14 minutes) and in the control group was 24 (20-29) minutes (P<0.001). The observers’ inter-rater agreement was 0.79.
Conclusion Skills in laparoscopic surgery can be increased in a clinically relevant manner using proficiency based virtual reality simulator training. The performance level of novices was increased to that of intermediately experienced laparoscopists and operation time was halved. Simulator training should be considered before trainees carry out laparoscopic procedures.
Trial registration ClinicalTrials.gov NCT00311792.
Laparoscopy has become the standard approach for many conditions in most surgical specialties.1 2 3 This development has been driven by the desire for less surgical trauma, faster postoperative recovery, shorter hospital stay, and better cosmetic results, and a sales drive by the medical industry.4 It is evident, however, that laparoscopy is associated with a longer operation time and a higher rate of surgical complications during the learning curve of the surgeons. This has been verified in many different specalties, including general5 6 urological,7 8 paediatric,9 and gynaecological surgery.10 The possibility of overcoming these problems during the learning curve by appropriate training and ensuring that surgeons perform a sufficient number of procedures has also been documented.11 The technical skills needed for laparoscopic surgery are fundamentally different from those for traditional open surgery, leading to a prolonged learning curve. The primary obstacles in learning laparoscopy are psychomotor and perceptual. The unique nature of laparoscopic surgery combined with an increasing focus on patients’ safety and rights, the present decrease in working hours, and concern over costs of operating theatre time are factors that challenge the traditional surgical approach and contribute to a growing need for novel methods in the training of laparoscopic surgeons.12 Although virtual reality simulation has the potential to offer important advantages in the area of training for new skills and procedures, evidence on the transfer of skills from the simulated environment to the operating theatre is still limited.13 14
We investigated the impact of training using a virtual reality simulator on the quality of skills acquired for a key gynaecological procedure. The investigation was carried out as a prospective, randomised, observer blinded, controlled trial, according to the guidelines of the consolidated standards of reporting trials (www.consort-statement.org).
From September 2006 to June 2007, 32 trainees in gynaecological specialty training years 1 and 2 (postgraduate years 3-8; see box), with no experience of advanced laparoscopy (defined as all laparoscopic procedures involving coordination of more than one instrument), were included in the study. Of a total cohort of 42 (38 women and four men) trainees in the region, eight were not eligible, as they were too experienced and four came from the two departments that did not participate in the trial. Of the remaining 30 eligible trainees, the first 24 who volunteered were enrolled. They came from seven of nine gynaecology departments in the Zeeland region of Denmark (population 2.3 million): Gentofte hospital (five trainees), Herlev hospital (n=4), Roskilde hospital (n=4), Hilleroed hospital (n=1), Holbaek hospital (n=1), Hvidovre hospital (n=2), and Rigshospitalet hospital (n=7).
Duration of specialist training in Denmark
Preregistration house year—one year
Introduction before specialist training—one year
Optional additional training to qualify for further specialisation (possibility for writing PhD thesis)—1-3 years
Specialist training—four years
Randomisation and blinding
To ensure that the trainees’ baseline characteristics were similar within and between each group, we chose a stratified randomisation based on previous experience of simple laparoscopy (defined as laparoscopic procedures performed using a single instrument, such as diagnostic laparoscopic sterilisation (clips) or diagnostic laparoscopy). The Clinical Trial Unit at Copenhagen University independently randomised the trainees by computer to intervention or control groups. The randomisation procedure was concealed and achieved by using the trainees’ unique personal identification number (central personal register number).
Trainees in the intervention group were given an oral introduction to the simulator and the rating scale used for outcome measure. Any operations done during the study were recorded. Owing to the nature of the trial it was not possible to blind the trainees to their allocated group, but all involved departments, supervisors, and staff in the operating theatres were blinded to the trainee’s group, and the assessors of outcome were blinded to both the trainee and their allocated group. The primary investigator saw the data only after completion of all assessments and once data had been loaded in a database.
The control group was to continue standard clinical education. During the study no trainee in either group was allowed to perform advanced laparoscopic surgery, only simple laparoscopy or to assist senior colleagues. To check that randomisation had been successful, the control group were trained in the simulator after the trial. Their baseline performances were indistinguishable from those of the intervention group.
The virtual reality laparoscopy simulator program (LapSim Gyn v 3.0.1; Surgical Science, Gothenburg, Sweden) was run on an IBM T42 computer in a docking station (Pentium M 1.8 GHz/512 MB RAM; IBM, Armonk, NY, USA) using an interface with a diathermy pedal (Virtual Laparoscopic Interface; Immersion, San Jose, CA, USA). The operations took place in the operating theatres of the participating departments and were recorded on DVD using a camera attached to the laparoscope for later blinded evaluation. During the operation one of the authors (CRL or designated TD) observed the procedure to record the handling of the surgical instruments, any involvement of the supervisor, whether the standard procedure for the operation was followed, and whether the recording was done correctly, finalised, and assessed.
The intervention group undertook a specific training programme in the simulator. The programme comprised two parts: firstly, training in the two basic skills of “lifting and grasping” and “cutting” during which the trainees were introduced to the simulator environment and the different instruments; secondly, one procedure specific task in which the trainee had to carry out a complete right sided salpingectomy while preserving the ovary. The training in basic skills was done once in each training cycle of 45-60 minutes and the salpingectomy repeated continually during the remainder of the cycle. The simulator provided the trainees with instant feedback on time, path length and angular path of the instruments’ movements, bleeding, cutting of uncoagulated arteries, and use of diathermy on non-target tissue. The training sessions were repeated until the expert criterion level was reached in two consecutive and independent simulations. The proficiency criteria were established by experts in previous studies of construct validity and learning curves.15 16 The requirements and settings of the simulator are available at www.skopisimulator.rh.dk.
The trainees performed their first salpingectomy at their local gynaecological department and were supervised by a senior colleague who was told about the purpose of the trial. To make comparison of performance easier, the trainees all carried out procedures on the right side. The patients were admitted for elective salpingectomy before treatment for infertility or for prophylactic removal of fallopian tubes and ovaries owing to a positive test result for breast cancer gene 1 (BRCA1). The trainees were not allowed to operate on patients who had undergone previous open or laparoscopic surgery below umbilicus, had possible abdominal malignant disease, had an American Surgical Association score ≥3 (patients with severe systemic disease), had a body mass index less than 18 or more than 27, had haemophilia, or had other factors of potential influence on the surgical procedure. The operations followed a modified standard procedure on the basis of expert consensus.17 18 The supervisors were allowed to give oral instructions only, and one researcher was present to observe the procedure and to record who handled the instruments.
The primary outcome measure was technical performance, measured as total score (10-50 points) using the objective structured assessment of laparoscopic salpingectomy, which comprises a five item general rating scale and five item task specific rating scale.19 Two independent observers blinded to trainee and allocated group assessed the recorded operations. The secondary outcome measure was operating time in minutes. The reliability of the assessment was determined by calculating the inter-rater agreement (number of agreements for each of the assessed items divided by total number of assessed items) and the γ coefficient. We present outcomes as medians and interquartile ranges.
The power calculation was based on a previous validation study on the procedure specific scale of the objective structured assessment of laparoscopic salpingectomy.19 This study showed a difference of six points between novice laparoscopists (0-5 procedures) and intermediately experienced laparoscopists (30-50 procedures). An improvement of skills to the level of 30 or more points was considered acceptable. On the basis of these findings we chose the minimal relevant difference to be six points. We determined that with an α of 0.05 (two sided) and a power of 80% (β=0.2 giving Zα=1.96 and Zβ=0.84, largest SD=4.40) we required 18 or more trainees. To compensate for possible drop outs, we added an additional third to the 18, totalling 24 trainees.
We present cumulated scores as medians (average score of two observers), compared using non-parametrical analysis (Mann-Whitney U test). We considered a two tailed P value of 0.05 or less to be statistically significant and an inter-rater agreement and γ coefficient of 0.8 or more for each to be acceptable. Analysis was done using SPSS 13.0 for Windows. Graphics were made using Graph pad Prism (Graph Pad, San Diego, CA, USA).
Eight of the total cohort of 42 trainees (38 women, four men) were ineligible for the study as they were too experienced and four came from the two departments not participating in the trial. The remaining 30 trainees agreed to participate. The first 24 were enrolled; 22 (90%) were women, representing the current sex distribution among trainees in obstetrics and gynaecology in Denmark (figure⇓). The average age of the trainees was 32.8 years (range 26-42 years), and 23 were right handed. Eleven trainees were randomised to virtual reality training in laparoscopic salpingectomy and 10 were randomised to traditional clinical education. Table 1⇓ shows the baseline characteristics of the trainees. Two trainees were subsequently excluded from the simulator trained group because one failed to complete the training programme and the other was involved in an operation that was cancelled because of anatomical abnormality and suspected malignant disease in the patient. One trainee was excluded from the control group because of a technical fault in the DVD recorder used to record the operation.
|Characteristics||Simulator trained group (n=13)||Control group (n=11)|
|Mean (range) age (years)||33.3 (30-42)||32.4 (26-38)|
|Experience of simple laparoscopy||6||5|
|No experience of simple laparoscopy||7||6|
The median total score on the general and task specific rating scale reached 33 points (interquartile range 32-36 points) in the simulator trained group and 23 (22-27 points) in the control group (P<0.001, table 2⇓).
|Outcome measure||Simulator trained group (n=11)||Control group (n=10)||P value*|
|Total score (points)||33 (25-39; 32-36)||23 (21-28; 22-27)||<0.001|
|% reaching ≥30 points||82||0|
|Total time (minutes)||12 (6-24; 10-14)||24 (14-38; 20-29)||<0.001|
Inter-rater agreement 0.79. γ-coefficient 0.83 (95% confidence interval 0.68 to 0.98).
*Mann-Whitney U test.
The median total time to complete the procedure was 12 minutes (interquartile range 10-14 minutes) in the simulator trained group compared with 24 (20-29 minutes) in the control group (P<0.001, table 2). Twenty one operations were assessed.
The median number of simulated salpingectomies needed to reach the proficiency level in the simulator trained group was 28 (24-32 salpingectomies). The control group was offered simulator training after the study operation; nine of the 11 trainees in this group volunteered and a median of 26 (23-32) simulated operations were needed to reach the proficiency level (P=0.70). The mean time spent on training using the simulator was 7 hours and 15 minutes (5h 30 min-8h 0 min) in the intervention group and 7 hours and 0 min (5h 15 min-7h 45 min) in the control group (P=0.65; table 3⇓). The baseline score (first attempt) was 8 (5-15) in the simulator trained group and 9 (7-19) in the control group after training (P=0.70; table 3). All trends of differences in baseline characteristics were not statistically significant.
|Variable||Simulator trained group (n=11)||Control group (n=9)*||P value†|
|No (range) of training sessions||28 (16-39)||6 (19-43)||0.76|
|Duration (range) of training||7h 15m (4h 15m-9h 30m)||7h 0m (4h 0m-9h 15m)||0.70|
|Median (range) score on first attempt (%)||8 (5-15)||9 (7-19)||—|
*Voluntary simulator training after surgery.
†Mann-Whitney U test.
The time used by the assessors to fill in the rating chart was the mean total operation time plus five minutes for each DVD recording. The inter-rater agreement was 0.79 (166/210). The γ coefficient used to investigate strength of correlations among the observers at single subject level reached 0.83 (95% confidence interval 0.69 to 0.99).
Proficiency based virtual reality training in laparoscopic salpingectomy compared with standard clinical education was associated with a clinically important improvement of operative skills during the actual procedure. The learning curve in the operating theatre was also shorter. On the rating scale used in this study, which had previously been validated in a separate investigation, novices (fewer than five procedures) scored a median 24 points, and intermediately experienced trainees (20-50 procedures) a median 33 points compared with a median 39 points for experts.19 The clinical implications of the present findings are thus extensive. After training in a specific procedure to a predefined (proficiency based) level inexperienced trainees progressed from the performance level of a novice to that of an intermediately experienced gynaecologist, assessed in their first complex laparoscopic procedure. By using simulator training it might be possible to bypass the early learning curve, which is known to be associated with an increased rate of complications.20 This study was not designed to investigate complication rates, and conclusions in this area must be drawn cautiously. In general it is difficult to use patient outcomes to evaluate a medical training course. Firstly, in contrast with trials of a single intervention (for example, a new drug) medical education is a complex intervention involving many interconnecting parts and different layers.21 Secondly, assessment of surgical technical skills of individual trainees will need to be based on surrogate end points rather than outcomes such as morbidity or mortality because it is an ethical imperative that an operation performed by a supervised novice ought to have the same outcome as that of the supervisor. Training may cost time and some inconvenience for the patient but should never jeopardise safety or outcome. Thirdly, to show differences in outcome, based on a training course, the numbers of trainees should by far outnumber the total number of trainees available, thus making such a trial unfeasible.
Although operating time might be greater with novice surgeons, the outcomes of a supervised operation ought to be the same. The time to complete the laparoscopic salpingectomy was reduced by half. As the operating theatre serves both productivity and educational purposes, shorter operation times are of benefit.
The present results emphasise that by using virtual reality simulator training the surgical community can meet the need for proficiency based basic training in laparoscopy. These results also show that criterion based procedural training using a virtual reality simulator can help compensate for reduced working hours by bringing trainees to a higher level of performance more quickly. Traditional training depends on the supply of suitable procedures for training purposes, whereas simulator training can be used according to demand. To achieve an average of 28 salpingectomies can take a year or more in clinical practice, compared with eight hours of intensive training using the simulator.
Finally, reducing the operating time by half, from 24 minutes in the control group to 12 minutes in the simulator trained group, might not seem important. Nevertheless it has been estimated that during residency the additional costs of longer operating time is about (1997) $48 000 (£31 841.00; €35 907.00) per graduate.22 For more extensive surgery or just a bilateral procedure, increased use of simulator training could reduce novice operating time substantially, easing the pressure on the often limited capacity for training in the operating theatre.
Transfer of skills
Despite virtual reality simulators being introduced more than a decade ago, evidence on the effect of simulators on performance in the operating theatre is still sparse.13 To date no published studies on the transfer of technical surgical skills from simulator to real operations had exceeded grade 2a evidence; in our study the level of evidence is 1b. These findings are supported in a systematic review,14 which also showed that the randomised controlled trials on effect of simulator training were generally of poor scientific quality. The major problems were randomisation with inadequate concealment of allocation, lack of power calculations or insufficient sample sizes, lack of observer blinding, insufficient training time or goals, non-standardised comparators in control groups, and disparate interventions in intervention groups. Most studies also used surrogate end points such as bench station or simulator tests to establish the transferability of skills, instead of assessing skills in real operations. The conclusion of the meta-analysis was that only few studies possess the necessary quality, that two studies showed a positive effect23 24 (real operation) and one study no effect25 (simulated operation) of simulator based training. Another common feature of the previous studies is that they were all carried out using basic skills rather than procedure specific simulation. We used a procedural simulator, which provides training in both psychomotor and cognitive skills, thereby also improving knowledge of the surgical procedure and its potential pitfalls. There are probably several reasons for the significant impact on both performance and time in our study compared with previously published studies. Firstly, the simulator provides a realistic graphic presentation of anatomy in the surgical field and a good feedback system enabling the trainees to adjust their performance. Secondly, using predefined training goals (the expert proficiency level) instead of fixed time or numbers, encouraged the trainees to rehearse until they reached the maximal effect of the simulator training. Thirdly, instead of selecting medical students who might not choose a surgical career, we studied highly motivated trainees who needed to learn laparoscopic skills for the future.
We measured the impact of simulator training on a salpingectomy, which is an important gynaecological laparoscopic procedure and can be viewed as a key operation possessing all the core skills needed for most laparoscopic procedures. We did not test external validity and reproducibility beyond the specialty of gynaecology. However the skills that were improved can probably be considered as genuine core skills, which would be beneficial across the surgical specialties.
The flow of an operation is based on both technical skills and knowledge of the procedure, and therefore we expect the impact of simulator training would apply to other laparoscopic procedures, although the effect would be more visible on the technical side than in the knowledge of procedural skills. These suppositions are supported by a contemporary Swedish study on procedural virtual reality simulator training of cholecystectomy, which reached conclusions similar to those of the present study.26
In the present study the γ coefficient showed that there was no systematic discrepancy among the raters. The inter-rater agreement also reached a sufficient level, which is evidence of a valid and reliable assessment.
The investigation was carried out in the same way that a curriculum integrated training course most likely would be implemented. The internal consistency of the trial could have been higher if all the trainees had operated in the same theatre, using the same technical equipment, and with the same supervisor and staff. However, by showing the effects of simulator training in settings closely resembling a regional simulator training course the external validity was improved. The primary investigator helped the trainee to use the simulator and introduced the different training modules but did not teach laparoscopic techniques. The feedback on performance was based on assessment in the simulator. A designated supporter at the training session could, however, be a source of bias. A setting where trainees practise by themselves could eliminate this potential source of bias. Finally, performing laparoscopic surgery also consists of identifying diseased anatomy, communication, teamwork, decision making,27 and leadership, alternative plans, and conversion to open surgery if needed.28 29 These non-technical skills are taught in the currently existing virtual reality systems to a limited degree only. In this study we did not provide training in these non-technical skills or assess them. Simulator training should probably be considered only as a supplement or preoperative training; further education and practice in the operating theatre as well as further development of more complex virtual surgical environments (hybrid simulation)30 is still required.
It is possible to transfer skills acquired during proficiency based training using a virtual reality simulator to a real operation. Training in proficiency based skills should be incorporated in a comprehensive surgical training and assessment curriculum for all residents before they operate on real patients. This can potentially improve patients’ safety and improve efficiency in the operating theatre.
What is already known on this topic
The European Working Time Directive has put extra pressure on surgical training programmes
Virtual reality simulators could contribute to the training of core skills for laparoscopy
High grade evidence of the effect of virtual reality simulator training on real operations is sparse
What this study adds
Training using a virtual reality simulator improved performance in a laparoscopic procedure
Cite this as: BMJ 2009;338:b1802
We thank Jorn Wetterslev at the Copenhagen Trials Unit for critical revision of the research protocol, assistance with the power calculation, and the concealed computer based randomisation of the participants.
Contributors: CRL (principal investigator) acquired the data, drafted the paper, did the statistical analysis, and obtained funding. TPG and JLS provided administrative support and critically revised the manuscript. JLS had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. TD acquired the data, provided technical support, and critically revised the manuscript. LS and CO acquired the data, provided administrative support, and critically revised the manuscript. TVS and BSO provided administrative support and supervised the study, critically revised the manuscript, and obtained funding. All authors conceived and designed the study and analysed and interpreted the data.
Funding: This project was supported by Copenhagen University Rigshospitalet Hospital. Trygfondet supplied various materials including computer hardware. Det Calssenske Fidecommis’ Jubilaeumsfond provided travel expenses. Aase and Ejner Danielsens foundation provided software maintenance and updates, DVD recorders, and a TV monitor. The Danish Society for the Protection of Laboratory Animals provided computer hardware and software. All phases of the present work including design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the final manuscript were done independent of the funders.
Competing interests: None declared.
Ethical approval: The investigation fully complied with the Helsinki II declaration on biomedical research. The study was approved by the Danish National Committee on Biomedical Research Ethics (approval code (KF) 01 28 37 56). All study participants and patients were provided with written study documentation and were included in the trial after informed consent. The Danish Data Protection Agency approved the collection, analysis, and storage of the DVD recordings (approval code: 2005-41-5817).
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.