SkinningGS: Editable Dynamic Human Scene Reconstruction Using Gaussian Splatting Based on a Skinning Model

TL;DR

This paper presents SkinningGS, a novel method for real-time, high-quality dynamic human scene reconstruction from monocular videos, leveraging Gaussian splatting and a skinning model for efficient and detailed avatar creation.

Contribution

It introduces a decoupled reconstruction framework using Gaussian splatting and a position texture on the SMPL model, enabling real-time, detailed, and resource-efficient human scene reconstruction.

Findings

Surpasses HUGS in reconstruction metrics.

Achieves over 100 FPS rendering speed.

Reduces GPU resource consumption during training.

Abstract

Reconstructing an interactive human avatar and the background from a monocular video of a dynamic human scene is highly challenging. In this work we adopt a strategy of point cloud decoupling and joint optimization to achieve the decoupled reconstruction of backgrounds and human bodies while preserving the interactivity of human motion. We introduce a position texture to subdivide the Skinned Multi-Person Linear (SMPL) body model's surface and grow the human point cloud. To capture fine details of human dynamics and deformations, we incorporate a convolutional neural network structure to predict human body point cloud features based on texture. This strategy makes our approach free of hyperparameter tuning for densification and efficiently represents human points with half the point cloud of HUGS. This approach ensures high-quality human reconstruction and reduces GPU resource consumption during training. As a result, our method surpasses the previous state-of-the-art HUGS in reconstruction metrics while maintaining the ability to generalize to novel poses and views. Furthermore, our technique achieves real-time rendering at over 100 FPS, $\sim$6$\times$ the HUGS speed using only Linear Blend Skinning (LBS) weights for human transformation. Additionally, this work demonstrates that this framework can be extended to animal scene reconstruction when an accurately-posed model of an animal is available.