A Continuous Optimization Approach for Efficient and Accurate Scene Flow

TL;DR

This paper introduces a continuous optimization framework for dense 3D scene flow estimation from stereo images, achieving high accuracy and efficiency by reformulating the problem into a purely continuous domain and leveraging a novel initialization and refinement process.

Contribution

It presents a novel continuous formulation for scene flow estimation that simplifies the optimization process and improves computational efficiency compared to previous discrete-continuous methods.

Findings

Ranked third on the KITTI Scene Flow benchmark

Achieved 3 to 30 times faster processing than top competitors

Demonstrated high accuracy with a purely continuous optimization approach

Abstract

We propose a continuous optimization method for solving dense 3D scene flow problems from stereo imagery. As in recent work, we represent the dynamic 3D scene as a collection of rigidly moving planar segments. The scene flow problem then becomes the joint estimation of pixel-to-segment assignment, 3D position, normal vector and rigid motion parameters for each segment, leading to a complex and expensive discrete-continuous optimization problem. In contrast, we propose a purely continuous formulation which can be solved more efficiently. Using a fine superpixel segmentation that is fixed a-priori, we propose a factor graph formulation that decomposes the problem into photometric, geometric, and smoothing constraints. We initialize the solution with a novel, high-quality initialization method, then independently refine the geometry and motion of the scene, and finally perform a global non-linear refinement using Levenberg-Marquardt. We evaluate our method in the challenging KITTI Scene Flow benchmark, ranking in third position, while being 3 to 30 times faster than the top competitors.