EAST: Environment-Aware Stylized Transition Along the Reality-Virtuality Continuum

TL;DR

This paper presents EAST, a framework that uses style transfer with 3D Gaussian Splatting to create seamless, aesthetically consistent transitions in VR environments during real-world interruptions, enhancing immersion.

Contribution

EAST introduces a novel style-transferred 3D Gaussian Splatting technique for seamless, aesthetic-aware transitions in VR, addressing limitations of previous digital replica methods.

Findings

Enhanced user comfort during transitions

Improved visual coherence across VR styles

Significant positive user feedback in qualitative studies

Abstract

In the Virtual Reality (VR) gaming industry, maintaining immersion during real-world interruptions remains a challenge, particularly during transitions along the reality-virtuality continuum (RVC). Existing methods tend to rely on digital replicas or simple visual transitions, neglecting to address the aesthetic discontinuities between real and virtual environments, especially in highly stylized VR games. This paper introduces the Environment-Aware Stylized Transition (EAST) framework, which employs a novel style-transferred 3D Gaussian Splatting (3DGS) technique to transfer real-world interruptions into the virtual environment with seamless aesthetic consistency. Rather than merely transforming the real world into game-like visuals, EAST minimizes the disruptive impact of interruptions by integrating real-world elements within the framework. Qualitative user studies demonstrate significant enhancements in cognitive comfort and emotional continuity during transitions, while quantitative experiments highlight EAST's ability to maintain visual coherence across diverse VR styles.