Fail-Safe Adversarial Generative Imitation Learning

TL;DR

This paper introduces a safe generative adversarial imitation learning framework with a safety layer that ensures actions are safe, providing theoretical safety guarantees and demonstrating effectiveness on real-world driver data.

Contribution

It presents a novel safety layer with a closed-form density and gradient, enabling safe end-to-end adversarial training with theoretical robustness guarantees.

Findings

The safety layer improves robustness during training.

The method achieves safe and effective imitation on real-world data.

Theoretical analysis confirms linear imitation error growth with horizon.

Abstract

For flexible yet safe imitation learning (IL), we propose theory and a modular method, with a safety layer that enables a closed-form probability density/gradient of the safe generative continuous policy, end-to-end generative adversarial training, and worst-case safety guarantees. The safety layer maps all actions into a set of safe actions, and uses the change-of-variables formula plus additivity of measures for the density. The set of safe actions is inferred by first checking safety of a finite sample of actions via adversarial reachability analysis of fallback maneuvers, and then concluding on the safety of these actions' neighborhoods using, e.g., Lipschitz continuity. We provide theoretical analysis showing the robustness advantage of using the safety layer already during training (imitation error linear in the horizon) compared to only using it at test time (up to quadratic error). In an experiment on real-world driver interaction data, we empirically demonstrate tractability, safety and imitation performance of our approach.