Hyperbolic Cosine Transformer for LiDAR 3D Object Detection

TL;DR

This paper introduces ChTR3D, a hyperbolic cosine transformer for LiDAR 3D object detection that reduces computational complexity and improves inference speed while maintaining competitive accuracy.

Contribution

The paper proposes a novel cosh-attention mechanism with linear complexity, enhancing efficiency in 3D object detection from LiDAR data.

Findings

Cosh-attention significantly speeds up inference.

ChTR3D outperforms vanilla attention in speed on KITTI.

It maintains competitive detection accuracy.

Abstract

Recently, Transformer has achieved great success in computer vision. However, it is constrained because the spatial and temporal complexity grows quadratically with the number of large points in 3D object detection applications. Previous point-wise methods are suffering from time consumption and limited receptive fields to capture information among points. In this paper, we propose a two-stage hyperbolic cosine transformer (ChTR3D) for 3D object detection from LiDAR point clouds. The proposed ChTR3D refines proposals by applying cosh-attention in linear computation complexity to encode rich contextual relationships among points. The cosh-attention module reduces the space and time complexity of the attention operation. The traditional softmax operation is replaced by non-negative ReLU activation and hyperbolic-cosine-based operator with re-weighting mechanism. Extensive experiments on the widely used KITTI dataset demonstrate that, compared with vanilla attention, the cosh-attention significantly improves the inference speed with competitive performance. Experiment results show that, among two-stage state-of-the-art methods using point-level features, the proposed ChTR3D is the fastest one.