A Mixed Reality System for Robust Manikin Localization in Childbirth Training

TL;DR

This paper presents a mixed reality system for childbirth training that combines virtual guidance with tactile feedback, improving localization accuracy and trainee performance compared to VR-only methods.

Contribution

The paper introduces a novel MR system with a spatially calibrated RGB-D camera and a coarse-to-fine localization pipeline for realistic childbirth training.

Findings

MR system achieves accurate, stable manikin localization on standalone headsets.

Trainees perform better with MR training than VR in delivery tasks.

MR training is preferred by medical students over VR-based methods.

Abstract

Opportunities for medical students to gain practical experience in vaginal births are increasingly constrained by shortened clinical rotations, patient reluctance, and the unpredictable nature of labour. To alleviate clinicians' instructional burden and enhance trainees' learning efficiency, we introduce a mixed reality (MR) system for childbirth training that combines virtual guidance with tactile manikin interaction, thereby preserving authentic haptic feedback while enabling independent practice without continuous on-site expert supervision. The system extends the passthrough capability of commercial head-mounted displays (HMDs) by spatially calibrating an external RGB-D camera, allowing real-time visual integration of physical training objects. Building on this capability, we implement a coarse-to-fine localization pipeline that first aligns the maternal manikin with fiducial markers to define a delivery region and then registers the pre-scanned neonatal head within this area. This process enables spatially accurate overlay of virtual guiding hands near the manikin, allowing trainees to follow expert trajectories reinforced by haptic interaction. Experimental evaluations demonstrate that the system achieves accurate and stable manikin localization on a standalone headset, ensuring practical deployment without external computing resources. A large-scale user study involving 83 fourth-year medical students was subsequently conducted to compare MR-based and virtual reality (VR)-based childbirth training. Four senior obstetricians independently assessed performance using standardized criteria. Results showed that MR training achieved significantly higher scores in delivery, post-delivery, and overall task performance, and was consistently preferred by trainees over VR training.