Interactive and immersive image-guided control of interventional manipulators with a prototype holographic interface

Cristina Marie Morales Mojica, Nikolaos V. Tsekos, Jose Daniel Velazco-Garcia, Haoran Zhao, Ioannis Seimenis, Ernst L. Leiss, Dipan J. Shah, Andrew Webb, Aaron T. Becker, Panagiotis Tsiamyrtzis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

The emerging potential of augmented reality (AR) to improve 3D medical image visualization for diagnosis, by immersing the user into 3D morphology is further enhanced with the advent of wireless head-mounted displays (HMD). Such information-immersive capabilities may also enhance planning and visualization of interventional procedures. To this end, we introduce a computational platform to generate an augmented reality holographic scene that fuses pre-operative magnetic resonance imaging (MRI) sets, segmented anatomical structures, and an actuated model of an interventional robot for performing MRI-guided and robot-assisted interventions. The interface enables the operator to manipulate the presented images and rendered structures using voice and gestures, as well as to robot control. The software uses forbidden-region virtual fixtures that alerts the operator of collisions with vital structures. The platform was tested with a HoloLens HMD in silico. To address the limited computational power of the HMD, we deployed the platform on a desktop PC with two-way communication to the HMD. Operation studies demonstrated the functionality and underscored the importance of interface customization to fit a particular operator and/or procedure, as well as the need for on-site studies to assess its merit in the clinical realm.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering, BIBE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1002-1005
Number of pages4
ISBN (Electronic)9781728146171
DOIs
StatePublished - Oct 1 2019
Event19th International Conference on Bioinformatics and Bioengineering, BIBE 2019 - Athens, Greece
Duration: Oct 28 2019Oct 30 2019

Other

Other19th International Conference on Bioinformatics and Bioengineering, BIBE 2019
CountryGreece
CityAthens
Period10/28/1910/30/19

Keywords

  • Augmented reality
  • Image guided interventions
  • Robot-assistance

ASJC Scopus subject areas

  • Information Systems
  • Biomedical Engineering
  • Health Informatics

Fingerprint Dive into the research topics of 'Interactive and immersive image-guided control of interventional manipulators with a prototype holographic interface'. Together they form a unique fingerprint.

Cite this