Improving the Intra-class Long-tail in 3D Detection via Rare Example Mining

TL;DR

This paper introduces a novel approach to enhance 3D object detection by mining rare examples based on feature space density estimation, significantly improving detection of infrequent, irregular objects.

Contribution

It proposes a new concept of rareness for data mining in 3D detection, utilizing flow models for density estimation and a cost-aware method to improve rare object detection.

Findings

Significant improvement in rare object detection accuracy (up to 30.97%).

Effective identification of rare objects using flow-based density estimation.

Enhanced overall performance of 3D detectors through targeted rare example mining.

Abstract

Continued improvements in deep learning architectures have steadily advanced the overall performance of 3D object detectors to levels on par with humans for certain tasks and datasets, where the overall performance is mostly driven by common examples. However, even the best performing models suffer from the most naive mistakes when it comes to rare examples that do not appear frequently in the training data, such as vehicles with irregular geometries. Most studies in the long-tail literature focus on class-imbalanced classification problems with known imbalanced label counts per class, but they are not directly applicable to the intra-class long-tail examples in problems with large intra-class variations such as 3D object detection, where instances with the same class label can have drastically varied properties such as shapes and sizes. Other works propose to mitigate this problem using active learning based on the criteria of uncertainty, difficulty, or diversity. In this study, we identify a new conceptual dimension - rareness - to mine new data for improving the long-tail performance of models. We show that rareness, as opposed to difficulty, is the key to data-centric improvements for 3D detectors, since rareness is the result of a lack in data support while difficulty is related to the fundamental ambiguity in the problem. We propose a general and effective method to identify the rareness of objects based on density estimation in the feature space using flow models, and propose a principled cost-aware formulation for mining rare object tracks, which improves overall model performance, but more importantly - significantly improves the performance for rare objects (by 30.97\%