Future imperfect

From cockpit to theatre

Virtual reality (VR) is a way for humans to visualise, manipulate, and interact with extremely complex data. Originally born out of the need to familiarise pilots with an aeroplane's instrumentation in the 1930s, as a form of mechanical flight simulation, virtual reality has followed a path of rapid technological development, with the major advances in digital simulation taking place since the 1980s. These have included everything from space flight simulators designed by NASA, to Fish Tank VR (a small virtual environment displayed on a computer monitor that tracks your head movements and shifts the angle of view and perspective accordingly).

However, the idea of applying it to medicine was developed about 15 years ago, by groups at the University of North Carolina and the US Department of Defense, who envisaged surgeons rehearsing complicated procedures using VR headsets.1

ERIC RISBERG/AP

A laparoscopic impulse device coupled with a virtual surgery simulator - now that's what I call science

• Download figure
• Open in new tab
• Download powerpoint

Unlike pure three dimensional visualisation, VR employs haptics. Robotic arms convey the relative sensations of pressure and force back to the instruments in the user's hands (also known as force feedback).

Every VR device is unique, but in general they all require a method for inputting the user's actions, for processing this information, and for feeding the results of the user's interaction within the virtual environment back to the user. Devices for learning the basics of how to control laparoscopic instruments provide good examples. The VR equipment transmits information gathered from position sensors located on or around the handheld surgical instruments, to a computer that transduces this information to an image of the instruments in a computer generated artificial environment, usually displayed on a computer monitor.

The VR environments are currently programmed in low level dedicated programming languages, such as the Virtual Reality Modelling Language (VRML), and run on powerful computers. The computers model how the graphical environments react to the user's movements based on either non-physics based models (for example, free form deformation), or on physics based models (for example, mass spring deformations). For example, if the latter is used, graphical environments respond more realistically when tissue is deformed by a (virtual) laparoscopic tool.

The image wire frames are surface rendered or texture mapped in real time and displayed either on a monitor, a head mounted display (to enhance three dimensional viewing), or, for maximum immersion, within a “cave” (a bit like an iMax cinema).

Remote surgery

Endoscopic surgery coupled with VR is one of the most rapidly developing areas. Endoscopic surgery is easier to simulate as surgeons already work with an unnatural view of the patient by looking at the monitor in front of them. In endoscopic surgery, access is already limited, tactile feedback restricted, and freedom of movement of instruments limited. Endoscopic surgery coupled with VR allows for operations to be performed remotely: in February this year, a woman in Ontario had laparoscopic antireflux surgery performed by a surgeon who was almost 400 km away, using telerobotic technology, which will eventually allow complex specialist surgery to be performed in many of the remote communities in Canada.2 Another fascinating area is that of virtual endoscopy. Data from conventional magnetic resonance imaging or computed tomography machines are combined into a virtual data model that can be explored by the surgeon as if an endoscope were inserted in the patient. Virtual colonoscopy has been shown to have a sensitivity of 90% and specificity of 94.6% in detecting lesions that are 10 mm or larger.3

VR is also being used in neurosurgery. It has been tried on one or two patients per week since 1999 and entails co-registering an image of the specific part of the brain that the surgeon is interested in and overlaying that on the patient. It is used preoperatively for planning and teaching, and intraoperatively for guidance, as shown in the figure.4 Several other potential uses for VR, in procedural training and competency assessment, have also been proposed or trialled—including hysteroscopy, amniocentesis, arthroscopy, epidural injection, and bronchoscopy5

Surgical education and training is probably the single most important application for VR in medicine—an area where you can only imagine the endless advantages. In the aviation industry, trainees hone their skills without the attendant dangers of flying, and studies have shown that two hours on the VR simulator are equivalent to one hour in the air. Evidence about the efficacy of such training measures in medicine has accumulated: endoscopic simulators such as the Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) and LapSim have been shown in randomised trials to improve laparoscopic skills in surgical trainees and medical students, respectively.67

Recently, basic virtual reality simulations have become available over the internet. Originally developed by Manchester University, they can be tried out at www.hoise.com/vmwc/projects/webset/articles/websetHome.html. These allow you to practise various practical skills, and the addition of a haptic mouse enhances the sense of reality. These systems are in the early stages of development, the full extent of which will realise the e-classroom, with e-mentors from sites around the world allowing the sharing of learning experiences.

A virtual walk

Potential for VR in this field is good because the demands of interaction and detailed visualisation are less stringent than for surgery. These systems simulate a physical environment; they can be used as an adjunct in the treatment of phobias, eating disorders, Parkinson's disease, and for the rehabilitation of stroke patients. A remarkable system from Japan, the “Bedside Wellness” system, allows bedridden patients to take a virtual forest walk while lying on their backs in bed. An array of three video screens presents the unfolding view of the forest as the patient gently steps on two foot pedals. To add even greater detail, there is three dimensional sound of birds, streams, and wind in the trees, and a slot below the central screen delivers a gentle breeze scented with pine to the “walking” patient.8

Back to reality

VR techniques currently lack odour and sound feedback. Also, force feedback is only crudely available. There has been some progress with the use of Phantom haptic feedback devices, which provide a degree of tactile feedback, and the development of bespoke haptic devices specifically for laparoscopic surgery. A need also exists to integrate data from several sources and to be able to switch instantaneously between the real and virtual worlds. More work is required to enable better simulation of the behaviours and characteristics of soft tissues.

Cyber sickness, similar to motion sickness, can have health consequences for VR users, such as fatigue and vertigo. However, these effects are usually mild and disappear quickly, and they are likely to become less important with the development of decreased time lag (between user actions and picture updates) and improved picture quality.9

Much of the technology used in VR is, of course, extremely expensive. However, VR may partially replace many current surgical training modalities, which are themselves expensive—for example, the acquisition and proper maintenance of cadaveric models for anatomical and surgical training. Moreover, increasing demand for VR simulators will surely bring down the cost to levels affordable to most hospitals in the developed world.

Using data acquired from a specific patient, VR will soon promise the ability to practice on your patient before treatment. The evolution and further development of haptics will allow you to touch virtual organs, sense their textures, and rehearse operative procedures in an immensely immersive environment. Improved fidelity, primarily with respect to the quality of graphics and their response to user input, as well as greater computing power and processor speeds will no doubt add to the realism. Enhanced programming will overcome problems such as accurately representing the random nature of bleeding and diathermy smoke, while incorporating patient specific pathology. The current cost of simulators (on average $100 000 (£54 000; €77 000) is a major problem, but in the future it is expected that VR machines will become more affordable as their popularity grows, following a similar trend to that of personal computers.