AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection

TL;DR

AutoShape introduces a shape-aware monocular 3D detection method that incorporates 2D/3D geometric constraints, significantly improving accuracy and achieving real-time performance on autonomous driving datasets.

Contribution

The paper presents a novel shape-aware 3D detection framework that integrates 2D keypoints and geometric constraints, enhancing detection accuracy over existing cuboid-based methods.

Findings

Significant performance improvement over baseline methods.

Achieves state-of-the-art results on KITTI dataset.

Operates in real-time for autonomous driving applications.

Abstract

Existing deep learning-based approaches for monocular 3D object detection in autonomous driving often model the object as a rotated 3D cuboid while the object's geometric shape has been ignored. In this work, we propose an approach for incorporating the shape-aware 2D/3D constraints into the 3D detection framework. Specifically, we employ the deep neural network to learn distinguished 2D keypoints in the 2D image domain and regress their corresponding 3D coordinates in the local 3D object coordinate first. Then the 2D/3D geometric constraints are built by these correspondences for each object to boost the detection performance. For generating the ground truth of 2D/3D keypoints, an automatic model-fitting approach has been proposed by fitting the deformed 3D object model and the object mask in the 2D image. The proposed framework has been verified on the public KITTI dataset and the experimental results demonstrate that by using additional geometrical constraints the detection performance has been significantly improved as compared to the baseline method. More importantly, the proposed framework achieves state-of-the-art performance with real time. Data and code will be available at https://github.com/zongdai/AutoShape