VR-prep improves visualization of medical data in augmented reality on smartphones

TL;DR

A new workflow called VR-prep improves the visualization of medical imaging data in augmented reality on smartphones using open-source tools.

Contribution

VR-prep is a novel pipeline that optimizes 3D rendering of medical image series in AR while reducing file size and preserving quality.

Findings

VR-prep significantly reduces file size for medical imaging data without sacrificing quality.

The workflow supports multiple imaging modalities including CT, MRI, US, and PET-CT.

VR-prep is user-friendly and facilitates broader integration of medical imaging data in AR.

Abstract

Relaying complex anatomical information between clinicians and to patients is challenging in medical routine and might be improved via direct 3D visualization of image series in augmented reality (AR) in portable devices. We devised a workflow for displaying medical imaging data in augmented reality on smartphones, relying solely on open-source products: ‘Medical Imaging XR’ (MIXR) application, ‘3D Slicer’ software, and ‘Fiji’. We designed a pipeline for image transformation and data harmonization (VR-prep) to expedite and optimize the 3D rendering of DICOM series in MIXR. VR-prep significantly reduced the size of image files to be uploaded to MIXR, without sacrificing quality and readability of images. Due to reduced file size, upload and download times onto the mobile device were significantly lower. Our workflow is suitable for CT, MRI, US and PET-CT, whereas MIXR alone only permits CT data. VR-prep is user-friendly and allows for broader integration of medical imaging data sets in AR, to facilitate clinician to clinician and clinician to patient communication. VR-prep relies exclusively in open-source software making it viable to adopt in a broad range of applications from clinical decision-making, patient communication, and medical education.

The online version contains supplementary material available at 10.1038/s41598-025-26856-7.

Figures