Li3DeTr: A LiDAR based 3D Detection Transformer
Gopi Krishna Erabati, Helder Araujo

TL;DR
Li3DeTr is an innovative end-to-end LiDAR-based 3D detection transformer that leverages sparse convolution and deformable attention, achieving state-of-the-art results in autonomous driving datasets.
Contribution
The paper introduces Li3DeTr, a novel transformer architecture for 3D detection from LiDAR data, with a unique cross-attention block and set-to-set loss, outperforming existing methods.
Findings
Achieves 61.3% mAP and 67.6% NDS on nuScenes
Surpasses state-of-the-art methods with NMS
Knowledge distillation slightly improves performance
Abstract
Inspired by recent advances in vision transformers for object detection, we propose Li3DeTr, an end-to-end LiDAR based 3D Detection Transformer for autonomous driving, that inputs LiDAR point clouds and regresses 3D bounding boxes. The LiDAR local and global features are encoded using sparse convolution and multi-scale deformable attention respectively. In the decoder head, firstly, in the novel Li3DeTr cross-attention block, we link the LiDAR global features to 3D predictions leveraging the sparse set of object queries learnt from the data. Secondly, the object query interactions are formulated using multi-head self-attention. Finally, the decoder layer is repeated number of times to refine the object queries. Inspired by DETR, we employ set-to-set loss to train the Li3DeTr network. Without bells and whistles, the Li3DeTr network achieves 61.3% mAP and 67.6% NDS surpassing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Li3DeTr: A LiDAR based 3D Detection Transformer· youtube
Taxonomy
TopicsAdvanced Neural Network Applications · Human Pose and Action Recognition · Multimodal Machine Learning Applications
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Softmax · Adam · Label Smoothing · Position-Wise Feed-Forward Layer · Dense Connections · Absolute Position Encodings · Layer Normalization
