Intensity Video Guided 4D Fusion for Improved Highly Dynamic 3D Reconstruction

TL;DR

This paper introduces a 4D fusion method guided by intensity videos to enhance high-speed 3D reconstruction of dynamic objects, effectively reducing noise and fluctuations in the data.

Contribution

The paper proposes a novel intensity video guided multi-frame 4D fusion pipeline that improves 3D reconstruction quality for highly dynamic scenes.

Findings

Effective noise reduction in 3D data

Robustness against intensity noise

Outperforms existing algorithms in dynamic scenarios

Abstract

The availability of high-speed 3D video sensors has greatly facilitated 3D shape acquisition of dynamic and deformable objects, but high frame rate 3D reconstruction is always degraded by spatial noise and temporal fluctuations. This paper presents a simple yet powerful intensity video guided multi-frame 4D fusion pipeline. Temporal tracking of intensity image points (of moving and deforming objects) allows registration of the corresponding 3D data points, whose 3D noise and fluctuations are then reduced by spatio-temporal multi-frame 4D fusion. We conducted simulated noise tests and real experiments on four 3D objects using a 1000 fps 3D video sensor. The results demonstrate that the proposed algorithm is effective at reducing 3D noise and is robust against intensity noise. It outperforms existing algorithms with good scalability on both stationary and dynamic objects.