ArticulatedFusion: Real-time Reconstruction of Motion, Geometry and Segmentation Using a Single Depth Camera

TL;DR

ArticulatedFusion introduces a real-time method for reconstructing dynamic scenes from a single depth camera, capturing motion, geometry, and segmentation simultaneously with improved robustness and efficiency.

Contribution

It presents a novel fusion-based approach with a segmentation-enhanced node graph and a two-level optimization for real-time articulated scene reconstruction.

Findings

Robust reconstruction of tangential and occluded motions.

Efficient node graph segmentation reduces optimization complexity.

Improved accuracy over previous dynamic scene reconstruction methods.

Abstract

This paper proposes a real-time dynamic scene reconstruction method capable of reproducing the motion, geometry, and segmentation simultaneously given live depth stream from a single RGB-D camera. Our approach fuses geometry frame by frame and uses a segmentation-enhanced node graph structure to drive the deformation of geometry in registration step. A two-level node motion optimization is proposed. The optimization space of node motions and the range of physically-plausible deformations are largely reduced by taking advantage of the articulated motion prior, which is solved by an efficient node graph segmentation method. Compared to previous fusion-based dynamic scene reconstruction methods, our experiments show robust and improved reconstruction results for tangential and occluded motions.