Control-Aware Prediction Objectives for Autonomous Driving

TL;DR

This paper introduces control-aware prediction objectives (CAPOs) that evaluate how predictions impact downstream control in autonomous driving, improving system safety and performance without requiring differentiable planners.

Contribution

The paper proposes novel control-aware prediction objectives that better align prediction training with control performance, enhancing autonomous vehicle safety and efficiency.

Findings

CAPOs improve system safety in suburban driving scenarios.

Using importance weights enhances prediction relevance for control.

Experiments in CARLA show better overall driving performance.

Abstract

Autonomous vehicle software is typically structured as a modular pipeline of individual components (e.g., perception, prediction, and planning) to help separate concerns into interpretable sub-tasks. Even when end-to-end training is possible, each module has its own set of objectives used for safety assurance, sample efficiency, regularization, or interpretability. However, intermediate objectives do not always align with overall system performance. For example, optimizing the likelihood of a trajectory prediction module might focus more on easy-to-predict agents than safety-critical or rare behaviors (e.g., jaywalking). In this paper, we present control-aware prediction objectives (CAPOs), to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories. Experimentally, we show our objectives improve overall system performance in suburban driving scenarios using the CARLA simulator.