VRGaussianAvatar: Integrating 3D Gaussian Avatars into VR

TL;DR

VRGaussianAvatar enables real-time, high-quality full-body 3D Gaussian avatars in VR using only HMD tracking, with efficient stereo rendering and improved user experience.

Contribution

It introduces a novel system integrating 3D Gaussian Splatting avatars into VR with a parallel pipeline and binocular batching for efficiency.

Findings

Achieves real-time performance in VR with 3D Gaussian avatars.

Provides higher perceived similarity and embodiment compared to mesh baselines.

Demonstrates effective stereo rendering with Binocular Batching.

Abstract

We present VRGaussianAvatar, an integrated system that enables real-time full-body 3D Gaussian Splatting (3DGS) avatars in virtual reality using only head-mounted display (HMD) tracking signals. The system adopts a parallel pipeline with a VR Frontend and a GA Backend. The VR Frontend uses inverse kinematics to estimate full-body pose and streams the resulting pose along with stereo camera parameters to the backend. The GA Backend stereoscopically renders a 3DGS avatar reconstructed from a single image. To improve stereo rendering efficiency, we introduce Binocular Batching, which jointly processes left and right eye views in a single batched pass to reduce redundant computation and support high-resolution VR displays. We evaluate VRGaussianAvatar with quantitative performance tests and a within-subject user study against image- and video-based mesh avatar baselines. Results show that VRGaussianAvatar sustains interactive VR performance and yields higher perceived appearance similarity, embodiment, and plausibility. Project page and source code are available at https://vrgaussianavatar.github.io.