FSHNet: Fully Sparse Hybrid Network for 3D Object Detection

TL;DR

FSHNet is a novel fully sparse hybrid network that improves 3D object detection by enhancing long-range feature extraction, optimizing training with dynamic label assignment, and refining details with a sparse upsampling module, validated on multiple benchmarks.

Contribution

Introduces SlotFormer for larger receptive fields, a dynamic sparse label assignment strategy, and a sparse upsampling module, advancing fully sparse 3D detection methods.

Findings

Outperforms existing methods on Waymo, nuScenes, and Argoverse2 benchmarks.

Enhances long-range feature extraction and small object detection.

Demonstrates significant improvements in detection accuracy.

Abstract

Fully sparse 3D detectors have recently gained significant attention due to their efficiency in long-range detection. However, sparse 3D detectors extract features only from non-empty voxels, which impairs long-range interactions and causes the center feature missing. The former weakens the feature extraction capability, while the latter hinders network optimization. To address these challenges, we introduce the Fully Sparse Hybrid Network (FSHNet). FSHNet incorporates a proposed SlotFormer block to enhance the long-range feature extraction capability of existing sparse encoders. The SlotFormer divides sparse voxels using a slot partition approach, which, compared to traditional window partition, provides a larger receptive field. Additionally, we propose a dynamic sparse label assignment strategy to deeply optimize the network by providing more high-quality positive samples. To further enhance performance, we introduce a sparse upsampling module to refine downsampled voxels, preserving fine-grained details crucial for detecting small objects. Extensive experiments on the Waymo, nuScenes, and Argoverse2 benchmarks demonstrate the effectiveness of FSHNet. The code is available at https://github.com/Say2L/FSHNet.