Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes

TL;DR

This paper investigates active learning for perception and prediction in self-driving cars, introducing a cost-aware, fine-grained selection method that improves model performance using partially labeled scenes.

Contribution

It proposes a novel, generalized active learning approach tailored for perception and prediction, enabling fine-grained, cost-effective data annotation in self-driving datasets.

Findings

Fine-grained selection improves perception accuracy

Cost-aware active learning reduces labeling effort

Enhances downstream planning performance

Abstract

Self-driving vehicles must perceive and predict the future positions of nearby actors in order to avoid collisions and drive safely. A learned deep learning module is often responsible for this task, requiring large-scale, high-quality training datasets. As data collection is often significantly cheaper than labeling in this domain, the decision of which subset of examples to label can have a profound impact on model performance. Active learning techniques, which leverage the state of the current model to iteratively select examples for labeling, offer a promising solution to this problem. However, despite the appeal of this approach, there has been little scientific analysis of active learning approaches for the perception and prediction (P&P) problem. In this work, we study active learning techniques for P&P and find that the traditional active learning formulation is ill-suited for the P&P setting. We thus introduce generalizations that ensure that our approach is both cost-aware and allows for fine-grained selection of examples through partially labeled scenes. Our experiments on a real-world, large-scale self-driving dataset suggest that fine-grained selection can improve the performance across perception, prediction, and downstream planning tasks.