Multimodal Cyber-physical Interaction in XR: Hybrid Doctoral Thesis Defense

TL;DR

This paper introduces a multimodal XR framework enabling seamless hybrid academic events, demonstrated through the first XR-based doctoral thesis defense, enhancing interaction and accessibility for remote and in-person participants.

Contribution

The paper presents a novel multimodal XR framework that supports hybrid academic events with full-body tracking and WebXR integration, enabling natural interaction across physical and virtual spaces.

Findings

Positive user feedback on VR experience

Effective support for hybrid event activities

Successful organization of the first XR doctoral defense

Abstract

Academic events, such as a doctoral thesis defense, are typically limited to either physical co-location or flat video conferencing, resulting in rigid participation formats and fragmented presence. We present a multimodal framework that breaks this binary by supporting a spectrum of participation - from in-person attendance to immersive virtual reality (VR) or browser access - and report our findings from using it to organize the first ever hybrid doctoral thesis defense using extended reality (XR). The framework integrates full-body motion tracking to synchronize the user's avatar motions and gestures, enabling natural interaction with onsite participants as well as body language and gestures with remote attendees in the virtual world. It leverages WebXR to provide cross-platform and instant accessibility with easy setup. User feedback analysis reveals positive VR experiences and demonstrates the framework's effectiveness in supporting various hybrid event activities.