MEDL-U: Uncertainty-aware 3D Automatic Annotation based on Evidential Deep Learning

TL;DR

MEDL-U introduces an uncertainty-aware evidential deep learning framework for 3D automatic annotation, improving pseudo label quality and uncertainty estimation to enhance 3D object detection accuracy.

Contribution

This paper presents the first evidential deep learning approach for 3D auto annotation that quantifies uncertainty and refines pseudo labels for better detection performance.

Findings

Probabilistic detectors trained with MEDL-U outperform deterministic ones.

MEDL-U achieves state-of-the-art results on KITTI test set.

Uncertainty estimation improves pseudo label quality and detection accuracy.

Abstract

Advancements in deep learning-based 3D object detection necessitate the availability of large-scale datasets. However, this requirement introduces the challenge of manual annotation, which is often both burdensome and time-consuming. To tackle this issue, the literature has seen the emergence of several weakly supervised frameworks for 3D object detection which can automatically generate pseudo labels for unlabeled data. Nevertheless, these generated pseudo labels contain noise and are not as accurate as those labeled by humans. In this paper, we present the first approach that addresses the inherent ambiguities present in pseudo labels by introducing an Evidential Deep Learning (EDL) based uncertainty estimation framework. Specifically, we propose MEDL-U, an EDL framework based on MTrans, which not only generates pseudo labels but also quantifies the associated uncertainties. However, applying EDL to 3D object detection presents three primary challenges: (1) relatively lower pseudolabel quality in comparison to other autolabelers; (2) excessively high evidential uncertainty estimates; and (3) lack of clear interpretability and effective utilization of uncertainties for downstream tasks. We tackle these issues through the introduction of an uncertainty-aware IoU-based loss, an evidence-aware multi-task loss function, and the implementation of a post-processing stage for uncertainty refinement. Our experimental results demonstrate that probabilistic detectors trained using the outputs of MEDL-U surpass deterministic detectors trained using outputs from previous 3D annotators on the KITTI val set for all difficulty levels. Moreover, MEDL-U achieves state-of-the-art results on the KITTI official test set compared to existing 3D automatic annotators.