Immersion into 3D Biomedical Data via Holographic AR Interfaces based on the Universal Scene Description (USD) Standard

Augmented reality (AR) enables immersion into imaging data, especially for visualization of 3-dimensional imaging data. High-quality and dynamic sceneries in AR platforms can positively impact many biomedical and bioinformatics fields, including education, training, research, and clinical practice. To create such sceneries, collaboration between multiple software platforms is crucial. While many file formats are capable of storing 3D models, 3D data sharing and collaboration between different applications to harness the power of each one in order to create high-quality multidimensional AR sceneries have always been challenging. It requires a common file format to enable real-time and efficient processing along the pipeline. Universal Scene Description (USD) is an open-source file format for robust and scalable interchange and augmentation of 3D scenes from various sources. It was also recently adopted by NVIDIA into its Omniverse platform and standardized by the Alliance for OpenUSD (AOUSD). The use of high-fidelity USD can be transformative in bioinformation for synergetic and interactive immersion of researchers and clinicians into 3D/4D data. AR immersion into USD data requires communication of servers or cloud facilities with the users' head-mounted display (HMD) or hand-held display (HHD) devices. We present a software application to bridge this gap between multi-modality AR immersion and USD data, which offers real-time multi-user AR immersion into USD data to enhance the potential of USD and take the AR immersion experience to the next level.