Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning

TL;DR

This paper presents a framework that uses multi-agent deep reinforcement learning to enable real-world driving control with minimal data, improving efficiency and generalization in autonomous vehicle navigation.

Contribution

It introduces a multi-agent RL approach with novel initialization and data augmentation techniques for real-world driving control, bridging the gap between simulation and reality.

Findings

Multi-agent RL outperforms single-agent in all tested scenarios.

Proposed techniques enable learning with minimal input data.

Successful deployment in TORCS virtual environment.

Abstract

Training self-driving cars is often challenging since they require a vast amount of labeled data in multiple real-world contexts, which is computationally and memory intensive. Researchers often resort to driving simulators to train the agent and transfer the knowledge to a real-world setting. Since simulators lack realistic behavior, these methods are quite inefficient. To address this issue, we introduce a framework (perception, planning, and control) in a real-world driving environment that transfers the real-world environments into gaming environments by setting up a reliable Markov Decision Process (MDP). We propose variations of existing Reinforcement Learning (RL) algorithms in a multi-agent setting to learn and execute the discrete control in real-world environments. Experiments show that the multi-agent setting outperforms the single-agent setting in all the scenarios. We also propose reliable initialization, data augmentation, and training techniques that enable the agents to learn and generalize to navigate in a real-world environment with minimal input video data, and with minimal training. Additionally, to show the efficacy of our proposed algorithm, we deploy our method in the virtual driving environment TORCS.