SMamba: Sparse Mamba for Event-based Object Detection

TL;DR

SMamba introduces an adaptive sparsification approach for event-based object detection, balancing accuracy and efficiency by selectively processing informative regions while maintaining global modeling capabilities.

Contribution

It proposes a novel adaptive sparsification method with modules for information assessment, local scanning, and global channel interaction, improving performance and efficiency over existing methods.

Findings

Outperforms other methods on three datasets in accuracy.

Reduces computational cost significantly.

Maintains high global modeling ability.

Abstract

Transformer-based methods have achieved remarkable performance in event-based object detection, owing to the global modeling ability. However, they neglect the influence of non-event and noisy regions and process them uniformly, leading to high computational overhead. To mitigate computation cost, some researchers propose window attention based sparsification strategies to discard unimportant regions, which sacrifices the global modeling ability and results in suboptimal performance. To achieve better trade-off between accuracy and efficiency, we propose Sparse Mamba (SMamba), which performs adaptive sparsification to reduce computational effort while maintaining global modeling capability. Specifically, a Spatio-Temporal Continuity Assessment module is proposed to measure the information content of tokens and discard uninformative ones by leveraging the spatiotemporal distribution differences between activity and noise events. Based on the assessment results, an Information-Prioritized Local Scan strategy is designed to shorten the scan distance between high-information tokens, facilitating interactions among them in the spatial dimension. Furthermore, to extend the global interaction from 2D space to 3D representations, a Global Channel Interaction module is proposed to aggregate channel information from a global spatial perspective. Results on three datasets (Gen1, 1Mpx, and eTram) demonstrate that our model outperforms other methods in both performance and efficiency.