SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatars

TL;DR

SqueezeMe introduces a lightweight framework that distills high-fidelity 3D Gaussian avatars into a mobile-friendly form, enabling real-time animation and rendering on VR headsets like Meta Quest 3.

Contribution

The paper presents a novel distillation method that simplifies Gaussian avatar models for mobile devices, achieving real-time performance without sacrificing visual quality.

Findings

Real-time rendering of Gaussian avatars at 72 FPS on Meta Quest 3.

Effective reduction of model parameters through pose correction sharing.

Successful demonstration of animation and rendering on mobile VR hardware.

Abstract

Gaussian-based human avatars have achieved an unprecedented level of visual fidelity. However, existing approaches based on high-capacity neural networks typically require a desktop GPU to achieve real-time performance for a single avatar, and it remains non-trivial to animate and render such avatars on mobile devices including a standalone VR headset due to substantially limited memory and computational bandwidth. In this paper, we present SqueezeMe, a simple and highly effective framework to convert high-fidelity 3D Gaussian full-body avatars into a lightweight representation that supports both animation and rendering with mobile-grade compute. Our key observation is that the decoding of pose-dependent Gaussian attributes from a neural network creates non-negligible memory and computational overhead. Inspired by blendshapes and linear pose correctives widely used in Computer Graphics, we address this by distilling the pose correctives learned with neural networks into linear layers. Moreover, we further reduce the parameters by sharing the correctives among nearby Gaussians. Combining them with a custom splatting pipeline based on Vulkan, we achieve, for the first time, simultaneous animation and rendering of 3 Gaussian avatars in real-time (72 FPS) on a Meta Quest 3 VR headset. Demo videos are available at https://forresti.github.io/squeezeme.