DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Diffusion Model

TL;DR

DifFlow3D introduces a diffusion probabilistic model-based approach for robust, uncertainty-aware scene flow estimation, significantly improving accuracy and reliability in dynamic 3D scene understanding.

Contribution

It presents a novel diffusion-based refinement framework with an uncertainty estimation module for scene flow, outperforming previous methods and enabling plug-and-play integration.

Findings

Achieves state-of-the-art performance with 24.0% and 29.1% EPE3D reduction on FlyingThings3D and KITTI datasets.

Attains millimeter-level accuracy (0.0078m EPE3D) on KITTI dataset.

Diffusion-based refinement can be integrated into existing networks to boost their accuracy.

Abstract

Scene flow estimation, which aims to predict per-point 3D displacements of dynamic scenes, is a fundamental task in the computer vision field. However, previous works commonly suffer from unreliable correlation caused by locally constrained searching ranges, and struggle with accumulated inaccuracy arising from the coarse-to-fine structure. To alleviate these problems, we propose a novel uncertainty-aware scene flow estimation network (DifFlow3D) with the diffusion probabilistic model. Iterative diffusion-based refinement is designed to enhance the correlation robustness and resilience to challenging cases, e.g. dynamics, noisy inputs, repetitive patterns, etc. To restrain the generation diversity, three key flow-related features are leveraged as conditions in our diffusion model. Furthermore, we also develop an uncertainty estimation module within diffusion to evaluate the reliability of estimated scene flow. Our DifFlow3D achieves state-of-the-art performance, with 24.0% and 29.1% EPE3D reduction respectively on FlyingThings3D and KITTI 2015 datasets. Notably, our method achieves an unprecedented millimeter-level accuracy (0.0078m in EPE3D) on the KITTI dataset. Additionally, our diffusion-based refinement paradigm can be readily integrated as a plug-and-play module into existing scene flow networks, significantly increasing their estimation accuracy. Codes are released at https://github.com/IRMVLab/DifFlow3D.