StreamMOS: Streaming Moving Object Segmentation with Multi-View Perception and Dual-Span Memory

TL;DR

StreamMOS introduces a streaming LiDAR-based moving object segmentation method with a dual-span memory mechanism and multi-view encoding, improving temporal consistency and accuracy in autonomous driving scenarios.

Contribution

The paper proposes a novel streaming network with short-term and long-term memory for consistent moving object segmentation in LiDAR sequences, incorporating multi-view encoding for enhanced feature extraction.

Findings

Achieves competitive results on SemanticKITTI and Sipailou Campus datasets.

Utilizes dual memory to improve temporal consistency in segmentation.

Employs multi-view encoder for better motion feature extraction.

Abstract

Moving object segmentation based on LiDAR is a crucial and challenging task for autonomous driving and mobile robotics. Most approaches explore spatio-temporal information from LiDAR sequences to predict moving objects in the current frame. However, they often focus on transferring temporal cues in a single inference and regard every prediction as independent of others. This may cause inconsistent segmentation results for the same object in different frames. To overcome this issue, we propose a streaming network with a memory mechanism, called StreamMOS, to build the association of features and predictions among multiple inferences. Specifically, we utilize a short-term memory to convey historical features, which can be regarded as spatial prior of moving objects and adopted to enhance current inference by temporal fusion. Meanwhile, we build a long-term memory to store previous predictions and exploit them to refine the present forecast at voxel and instance levels through voting. Besides, we present multi-view encoder with cascade projection and asymmetric convolution to extract motion feature of objects in different representations. Extensive experiments validate that our algorithm gets competitive performance on SemanticKITTI and Sipailou Campus datasets. Code will be released at https://github.com/NEU-REAL/StreamMOS.git.