CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Driving

TL;DR

CADRE introduces a novel cascade deep reinforcement learning framework for vision-based autonomous urban driving, combining offline perception training with online policy optimization to improve generalization and performance in complex urban environments.

Contribution

The paper proposes a new framework that integrates a co-attention perception module with distributed reinforcement learning for better urban driving performance.

Findings

CADRE outperforms state-of-the-art methods on CARLA NoCrash benchmark.

The co-attention perception module effectively captures visual-control relationships.

The framework demonstrates strong obstacle avoidance in urban scenarios.

Abstract

Vision-based autonomous urban driving in dense traffic is quite challenging due to the complicated urban environment and the dynamics of the driving behaviors. Widely-applied methods either heavily rely on hand-crafted rules or learn from limited human experience, which makes them hard to generalize to rare but critical scenarios. In this paper, we present a novel CAscade Deep REinforcement learning framework, CADRE, to achieve model-free vision-based autonomous urban driving. In CADRE, to derive representative latent features from raw observations, we first offline train a Co-attention Perception Module (CoPM) that leverages the co-attention mechanism to learn the inter-relationships between the visual and control information from a pre-collected driving dataset. Cascaded by the frozen CoPM, we then present an efficient distributed proximal policy optimization framework to online learn the driving policy under the guidance of particularly designed reward functions. We perform a comprehensive empirical study with the CARLA NoCrash benchmark as well as specific obstacle avoidance scenarios in autonomous urban driving tasks. The experimental results well justify the effectiveness of CADRE and its superiority over the state-of-the-art by a wide margin.