Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments

TL;DR

This paper introduces a reinforcement learning approach combining neural networks and Monte Carlo Tree Search to solve complex task and motion planning problems with LTL constraints in dynamic environments, demonstrated in autonomous driving simulations.

Contribution

It presents a novel hierarchical method integrating neural networks with tree search for planning under LTL constraints, addressing exploration challenges in complex environments.

Findings

Effective learning of control and task policies from LTL specifications

Successful navigation in simulated autonomous driving scenarios

Demonstrated ability to handle complex, dynamic environments

Abstract

We consider task and motion planning in complex dynamic environments for problems expressed in terms of a set of Linear Temporal Logic (LTL) constraints, and a reward function. We propose a methodology based on reinforcement learning that employs deep neural networks to learn low-level control policies as well as task-level option policies. A major challenge in this setting, both for neural network approaches and classical planning, is the need to explore future worlds of a complex and interactive environment. To this end, we integrate Monte Carlo Tree Search with hierarchical neural net control policies trained on expressive LTL specifications. This paper investigates the ability of neural networks to learn both LTL constraints and control policies in order to generate task plans in complex environments. We demonstrate our approach in a simulated autonomous driving setting, where a vehicle must drive down a road in traffic, avoid collisions, and navigate an intersection, all while obeying given rules of the road.