GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flow

TL;DR

GMA3D introduces a transformer-based module that leverages local and global semantic similarities to effectively estimate scene flow in occluded 3D point clouds, improving accuracy especially in real-world scenarios.

Contribution

This paper is the first to apply transformer architecture to address occlusion in scene flow estimation for point clouds, utilizing semantic self-similarity and motion consistency.

Findings

Achieved state-of-the-art results on the KITTI dataset for occluded scene flow.

Demonstrated effectiveness of GMA3D on non-occluded datasets like FlyThings3D.

Improved scene flow estimation accuracy in real-world occlusion scenarios.

Abstract

Scene flow represents the motion information of each point in the 3D point clouds. It is a vital downstream method applied to many tasks, such as motion segmentation and object tracking. However, there are always occlusion points between two consecutive point clouds, whether from the sparsity data sampling or real-world occlusion. In this paper, we focus on addressing occlusion issues in scene flow by the semantic self-similarity and motion consistency of the moving objects. We propose a GMA3D module based on the transformer framework, which utilizes local and global semantic similarity to infer the motion information of occluded points from the motion information of local and global non-occluded points respectively, and then uses an offset aggregator to aggregate them. Our module is the first to apply the transformer-based architecture to gauge the scene flow occlusion problem on point clouds. Experiments show that our GMA3D can solve the occlusion problem in the scene flow, especially in the real scene. We evaluated the proposed method on the occluded version of point cloud datasets and get state-of-the-art results on the real scene KITTI dataset. To testify that GMA3D is still beneficial to non-occluded scene flow, we also conducted experiments on non-occluded version datasets and achieved promising performance on FlyThings3D and KITTI. The code is available at https://anonymous.4open.science/r/GMA3D-E100.