PoseGaussian: Pose-Driven Novel View Synthesis for Robust 3D Human Reconstruction

TL;DR

PoseGaussian introduces a pose-guided Gaussian Splatting framework that enhances high-fidelity, real-time 3D human view synthesis by embedding pose information into geometric and temporal processing, addressing challenges like motion and occlusion.

Contribution

The paper presents a novel pose-guided Gaussian Splatting method that integrates pose signals into both geometric and temporal stages for improved robustness and generalization in dynamic human scene reconstruction.

Findings

Achieves real-time rendering at 100 FPS.

Outperforms prior methods in perceptual quality and structural accuracy.

Demonstrates state-of-the-art results on multiple datasets.

Abstract

We propose PoseGaussian, a pose-guided Gaussian Splatting framework for high-fidelity human novel view synthesis. Human body pose serves a dual purpose in our design: as a structural prior, it is fused with a color encoder to refine depth estimation; as a temporal cue, it is processed by a dedicated pose encoder to enhance temporal consistency across frames. These components are integrated into a fully differentiable, end-to-end trainable pipeline. Unlike prior works that use pose only as a condition or for warping, PoseGaussian embeds pose signals into both geometric and temporal stages to improve robustness and generalization. It is specifically designed to address challenges inherent in dynamic human scenes, such as articulated motion and severe self-occlusion. Notably, our framework achieves real-time rendering at 100 FPS, maintaining the efficiency of standard Gaussian Splatting pipelines. We validate our approach on ZJU-MoCap, THuman2.0, and in-house datasets, demonstrating state-of-the-art performance in perceptual quality and structural accuracy (PSNR 30.86, SSIM 0.979, LPIPS 0.028).