Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search

TL;DR

This paper introduces a decentralized cooperative planning method for automated vehicles using Hierarchical Monte Carlo Tree Search with macro-actions, enabling effective long-horizon planning and implicit cooperation in heterogeneous traffic environments.

Contribution

It presents a novel MCTS-based approach that models interdependent actions among traffic agents and learns macro-actions without predefined policies for improved cooperative planning.

Findings

Achieves effective cooperative planning in heterogeneous environments.

Demonstrates successful learning of macro-actions for longer-term planning.

Outperforms baseline methods in conflict scenarios.

Abstract

Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps and increase the effective search depth requiring fewer iterations to plan over longer horizons. Without predefined policies for macro-actions, the algorithm simultaneously learns policies over and within macro-actions. The proposed method is evaluated under several conflict scenarios, showing that the algorithm can achieve effective cooperative planning with learned macro-actions in heterogeneous environments.