Learning from Demonstration with Hierarchical Policy Abstractions Toward High-Performance and Courteous Autonomous Racing

TL;DR

This paper introduces a hierarchical learning framework for autonomous racing that combines demonstration-based trajectory prediction with residual control, achieving high performance and courteous interactions in multi-agent scenarios.

Contribution

It presents a novel hierarchical policy approach that learns from expert demonstrations to improve racing performance and courtesy in autonomous vehicles.

Findings

Outperforms baseline methods in simulation

Improves lap time and tracking accuracy

Balances performance with courteous driving

Abstract

Fully autonomous racing demands not only high-speed driving but also fair and courteous maneuvers. In this paper, we propose an autonomous racing framework that learns complex racing behaviors from expert demonstrations using hierarchical policy abstractions. At the trajectory level, our policy model predicts a dense distribution map indicating the likelihood of trajectories learned from offline demonstrations. The maximum likelihood trajectory is then passed to the control-level policy, which generates control inputs in a residual fashion, considering vehicle dynamics at the limits of performance. We evaluate our framework in a high-fidelity racing simulator and compare it against competing baselines in challenging multi-agent adversarial scenarios. Quantitative and qualitative results show that our trajectory planning policy significantly outperforms the baselines, and the residual control policy improves lap time and tracking accuracy. Moreover, challenging closed-loop experiments with ten opponents show that our framework can overtake other vehicles by understanding nuanced interactions, effectively balancing performance and courtesy like professional drivers.