Virtual and Mixed Reality Environments for Surgery and Therapy
Graduate Research Students: Jennifer, Cristian
As the evolution of surgery progresses towards minimally invasive procedures, visual access to target tissues are, in effect, becoming more limited. This is especially true for cardiac interventions such as valve repair or replacement, and for laparoscopic abdominal and pelvic procedures such as a cholecystectomy or prostatectomy.
In some cases, image guidance is reduced to a simple display of imaging data via a monitory in the operating room. The surgeon is then required to mentally reconstruct the 3D geometry of both surgical tools and patient anatomy.
Our Contribution
In response to these challenges, our group is integrating imaging data with various virtual elements to provide more context to the intra-operative images.
The global objective of our augmented reality (AR) platform is to provide the surgeon with a simple, intuitive system for surgical navigation in the absence of direct vision.
Figure 1. Schematic displaying the layout of the VR-enhanced US guidance surgical system. The integrated ultrasound and virtual reality data can be delivered to the surgeon either through a monitor or a head-mounted display (HMD).
Human factors research will be conducted to investigate any cognitive and physical constraints endured by a surgeon as he or she is presented with advanced medical imaging techniques during a simulation of a minimally invasive surgical procedure. These advanced visualization techniques include:
- Overlaying 3D virtual reality models of human organs with real-time ultrasound
- Augmenting real-time video images with medical data (ultrasound, MRI or CT images
- Using 3D stereoscopic visualization displays such as an HMD or a 2D monitor
By uncovering any human limitations while using these advanced visualization modalities, we can improve the usability of various interactive imaging methods designed for modern surgical procedures.
Key Questions
- Under what conditions can augmented image guidance improve visibility and coordination for the surgeon?
- What types of virtual or augmented images can benefit rather than hinder the surgery?
