Sequential Gaussian Avatars with Hierarchical Motion Context

TL;DR

SeqAvatar introduces a hierarchical motion context approach for neural rendering of 3D human avatars, improving detail capture and rendering speed over existing methods by integrating coarse-to-fine motion conditions and multi-scale sampling.

Contribution

The paper presents a novel hierarchical motion modeling framework that enhances 3DGS-based avatar rendering, achieving faster performance and better detail preservation compared to prior approaches.

Findings

Outperforms existing 3DGS-based methods in detail accuracy.

Runs significantly faster than NeRF-based models with temporal context.

Maintains or improves rendering quality in complex motion scenarios.

Abstract

The emergence of neural rendering has significantly advanced the rendering quality of 3D human avatars, with the recently popular 3DGS technique enabling real-time performance. However, SMPL-driven 3DGS human avatars still struggle to capture fine appearance details due to the complex mapping from pose to appearance during fitting. In this paper, we propose SeqAvatar, which excavates the explicit 3DGS representation to better model human avatars based on a hierarchical motion context. Specifically, we utilize a coarse-to-fine motion conditions that incorporate both the overall human skeleton and fine-grained vertex motions for non-rigid deformation. To enhance the robustness of the proposed motion conditions, we adopt a spatio-temporal multi-scale sampling strategy to hierarchically integrate more motion clues to model human avatars. Extensive experiments demonstrate that our method significantly outperforms 3DGS-based approaches and renders human avatars orders of magnitude faster than the latest NeRF-based models that incorporate temporal context, all while delivering performance that is at least comparable or even superior. Project page: https://zezeaaa.github.io/projects/SeqAvatar/