FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation

TL;DR

FlowNet3D++ introduces geometric constraints into deep scene flow estimation, significantly improving accuracy and reconstruction quality, and provides a new benchmark for evaluating dynamic 3D reconstruction performance.

Contribution

The paper proposes FlowNet3D++, incorporating geometric losses into deep scene flow estimation and introduces a new benchmark for dynamic 3D reconstruction evaluation.

Findings

Accuracy improved from 57.85% to 63.43%.

Reconstruction error reduced by up to 15%.

Achieves up to 35.2% improvement over previous methods.

Abstract

We present FlowNet3D++, a deep scene flow estimation network. Inspired by classical methods, FlowNet3D++ incorporates geometric constraints in the form of point-to-plane distance and angular alignment between individual vectors in the flow field, into FlowNet3D. We demonstrate that the addition of these geometric loss terms improves the previous state-of-art FlowNet3D accuracy from 57.85% to 63.43%. To further demonstrate the effectiveness of our geometric constraints, we propose a benchmark for flow estimation on the task of dynamic 3D reconstruction, thus providing a more holistic and practical measure of performance than the breakdown of individual metrics previously used to evaluate scene flow. This is made possible through the contribution of a novel pipeline to integrate point-based scene flow predictions into a global dense volume. FlowNet3D++ achieves up to a 15.0% reduction in reconstruction error over FlowNet3D, and up to a 35.2% improvement over KillingFusion alone. We will release our scene flow estimation code later.