DeFIX: Detecting and Fixing Failure Scenarios with Reinforcement Learning in Imitation Learning Based Autonomous Driving

TL;DR

DeFIX is a continuous learning framework that detects failure scenarios in imitation learning-based autonomous driving and trains reinforcement learning agents to fix these failures, improving safety and performance in urban environments.

Contribution

This paper introduces a novel iterative framework combining failure detection and reinforcement learning to enhance imitation learning in autonomous driving.

Findings

DeFIX outperforms state-of-the-art IL and RL benchmarks.

The framework effectively identifies failure scenarios in complex urban environments.

A single RL agent trained on failure scenarios can significantly improve driving safety.

Abstract

Safely navigating through an urban environment without violating any traffic rules is a crucial performance target for reliable autonomous driving. In this paper, we present a Reinforcement Learning (RL) based methodology to DEtect and FIX (DeFIX) failures of an Imitation Learning (IL) agent by extracting infraction spots and re-constructing mini-scenarios on these infraction areas to train an RL agent for fixing the shortcomings of the IL approach. DeFIX is a continuous learning framework, where extraction of failure scenarios and training of RL agents are executed in an infinite loop. After each new policy is trained and added to the library of policies, a policy classifier method effectively decides on which policy to activate at each step during the evaluation. It is demonstrated that even with only one RL agent trained on failure scenario of an IL agent, DeFIX method is either competitive or does outperform state-of-the-art IL and RL based autonomous urban driving benchmarks. We trained and validated our approach on the most challenging map (Town05) of CARLA simulator which involves complex, realistic, and adversarial driving scenarios. The source code is publicly available at https://github.com/data-and-decision-lab/DeFIX