Weakly Supervised Point Clouds Transformer for 3D Object Detection

TL;DR

This paper introduces a weakly supervised 3D object detection framework using a point clouds transformer, reducing annotation costs while achieving high accuracy on KITTI datasets.

Contribution

It proposes an Unsupervised Voting Proposal Module and combines transformer-based global features with ResNet local features for effective weakly supervised learning.

Findings

Achieved state-of-the-art average precision on KITTI dataset.

Effectively reduces supervision requirements for 3D object detection.

Combines transformer and ResNet for comprehensive feature extraction.

Abstract

The annotation of 3D datasets is required for semantic-segmentation and object detection in scene understanding. In this paper we present a framework for the weakly supervision of a point clouds transformer that is used for 3D object detection. The aim is to decrease the required amount of supervision needed for training, as a result of the high cost of annotating a 3D datasets. We propose an Unsupervised Voting Proposal Module, which learns randomly preset anchor points and uses voting network to select prepared anchor points of high quality. Then it distills information into student and teacher network. In terms of student network, we apply ResNet network to efficiently extract local characteristics. However, it also can lose much global information. To provide the input which incorporates the global and local information as the input of student networks, we adopt the self-attention mechanism of transformer to extract global features, and the ResNet layers to extract region proposals. The teacher network supervises the classification and regression of the student network using the pre-trained model on ImageNet. On the challenging KITTI datasets, the experimental results have achieved the highest level of average precision compared with the most recent weakly supervised 3D object detectors.