Towards High Efficient Long-horizon Planning with Expert-guided Motion-encoding Tree Search

TL;DR

This paper introduces EMTS, an enhanced tree search algorithm for autonomous driving that leverages expert policies and motion primitives to improve long-horizon planning efficiency.

Contribution

The paper proposes EMTS, extending MuZero with a motion primitives latent space and expert guidance to reduce search depth and improve efficiency in autonomous driving planning.

Findings

EMTS outperforms four algorithms in challenging scenarios.

EMTS achieves early convergence in training.

EMTS demonstrates improved search efficiency and effectiveness.

Abstract

Autonomous driving holds promise for increased safety, optimized traffic management, and a new level of convenience in transportation. While model-based reinforcement learning approaches such as MuZero enables long-term planning, the exponentially increase of the number of search nodes as the tree goes deeper significantly effect the searching efficiency. To deal with this problem, in this paper we proposed the expert-guided motion-encoding tree search (EMTS) algorithm. EMTS extends the MuZero algorithm by representing possible motions with a comprehensive motion primitives latent space and incorporating expert policies toimprove the searching efficiency. The comprehensive motion primitives latent space enables EMTS to sample arbitrary trajectories instead of raw action to reduce the depth of the search tree. And the incorporation of expert policies guided the search and training phases the EMTS algorithm to enable early convergence. In the experiment section, the EMTS algorithm is compared with other four algorithms in three challenging scenarios. The experiment result verifies the effectiveness and the searching efficiency of the proposed EMTS algorithm.