CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer

TL;DR

This paper introduces CT3D++ and CT3D, innovative frameworks for 3D object detection from point clouds that leverage keypoint-induced channel-wise transformers to improve accuracy and efficiency with minimal manual design.

Contribution

The paper presents CT3D++ with geometric and semantic fusion embedding and a point-to-key encoder, advancing 3D detection performance and computational efficiency.

Findings

Achieves state-of-the-art results on KITTI and Waymo datasets.

Introduces a novel keypoint-induced channel-wise transformer architecture.

Reduces computational cost while improving detection accuracy.

Abstract

The field of 3D object detection from point clouds is rapidly advancing in computer vision, aiming to accurately and efficiently detect and localize objects in three-dimensional space. Current 3D detectors commonly fall short in terms of flexibility and scalability, with ample room for advancements in performance. In this paper, our objective is to address these limitations by introducing two frameworks for 3D object detection with minimal hand-crafted design. Firstly, we propose CT3D, which sequentially performs raw-point-based embedding, a standard Transformer encoder, and a channel-wise decoder for point features within each proposal. Secondly, we present an enhanced network called CT3D++, which incorporates geometric and semantic fusion-based embedding to extract more valuable and comprehensive proposal-aware information. Additionally, CT3D ++ utilizes a point-to-key bidirectional encoder for more efficient feature encoding with reduced computational cost. By replacing the corresponding components of CT3D with these novel modules, CT3D++ achieves state-of-the-art performance on both the KITTI dataset and the large-scale Way\-mo Open Dataset. The source code for our frameworks will be made accessible at https://github.com/hlsheng1/CT3D-plusplus.