BIDA: A Bi-level Interaction Decision-making Algorithm for Autonomous Vehicles in Dynamic Traffic Scenarios

TL;DR

This paper introduces BIDA, a novel bi-level decision-making algorithm for autonomous vehicles that combines interactive Monte Carlo tree search with deep reinforcement learning to improve safety, efficiency, and interaction in dynamic traffic scenarios.

Contribution

The paper presents a new bi-level interaction decision-making framework integrating MCTS and DRL, specifically designed for complex traffic environments, which outperforms existing benchmarks.

Findings

Enhances interaction rationality and safety in AV decision-making.

Reduces computational costs compared to existing methods.

Outperforms benchmarks in safety, efficiency, and interaction in traffic scenarios.

Abstract

In complex real-world traffic environments, autonomous vehicles (AVs) need to interact with other traffic participants while making real-time and safety-critical decisions accordingly. The unpredictability of human behaviors poses significant challenges, particularly in dynamic scenarios, such as multi-lane highways and unsignalized T-intersections. To address this gap, we design a bi-level interaction decision-making algorithm (BIDA) that integrates interactive Monte Carlo tree search (MCTS) with deep reinforcement learning (DRL), aiming to enhance interaction rationality, efficiency and safety of AVs in dynamic key traffic scenarios. Specifically, we adopt three types of DRL algorithms to construct a reliable value network and policy network, which guide the online deduction process of interactive MCTS by assisting in value update and node selection. Then, a dynamic trajectory planner and a trajectory tracking controller are designed and implemented in CARLA to ensure smooth execution of planned maneuvers. Experimental evaluations demonstrate that our BIDA not only enhances interactive deduction and reduces computational costs, but also outperforms other latest benchmarks, which exhibits superior safety, efficiency and interaction rationality under varying traffic conditions.