SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selection

TL;DR

SF2SE3 is a new method that segments scene flow into independently moving objects with SE(3)-motions, improving accuracy in segmentation and odometry while maintaining competitive scene flow estimation.

Contribution

It introduces a novel sampling and selection strategy for SE(3)-motion proposals, enhancing scene segmentation and motion estimation accuracy.

Findings

Performs on par with state-of-the-art in scene flow estimation.

Achieves higher accuracy in object segmentation.

Improves visual odometry results.

Abstract

We propose SF2SE3, a novel approach to estimate scene dynamics in form of a segmentation into independently moving rigid objects and their SE(3)-motions. SF2SE3 operates on two consecutive stereo or RGB-D images. First, noisy scene flow is obtained by application of existing optical flow and depth estimation algorithms. SF2SE3 then iteratively (1) samples pixel sets to compute SE(3)-motion proposals, and (2) selects the best SE(3)-motion proposal with respect to a maximum coverage formulation. Finally, objects are formed by assigning pixels uniquely to the selected SE(3)-motions based on consistency with the input scene flow and spatial proximity. The main novelties are a more informed strategy for the sampling of motion proposals and a maximum coverage formulation for the proposal selection. We conduct evaluations on multiple datasets regarding application of SF2SE3 for scene flow estimation, object segmentation and visual odometry. SF2SE3 performs on par with the state of the art for scene flow estimation and is more accurate for segmentation and odometry.