Long-term Planning by Short-term Prediction

TL;DR

This paper introduces a novel two-phase approach for long-term planning in autonomous driving, combining supervised learning for short-term prediction with recurrent neural networks for trajectory modeling, addressing challenges of continuous spaces and adversarial interactions.

Contribution

It proposes a decomposed planning framework that uses differentiable predictions and recurrent neural networks to handle non-Markovian, continuous, and adversarial environments in autonomous driving.

Findings

Effective long-term planning through supervised learning and RNNs

Robust policies learned with adversarial environment modeling

Addresses non-Markovian and continuous state-action challenges

Abstract

We consider planning problems, that often arise in autonomous driving applications, in which an agent should decide on immediate actions so as to optimize a long term objective. For example, when a car tries to merge in a roundabout it should decide on an immediate acceleration/braking command, while the long term effect of the command is the success/failure of the merge. Such problems are characterized by continuous state and action spaces, and by interaction with multiple agents, whose behavior can be adversarial. We argue that dual versions of the MDP framework (that depend on the value function and the $Q$ function) are problematic for autonomous driving applications due to the non Markovian of the natural state space representation, and due to the continuous state and action spaces. We propose to tackle the planning task by decomposing the problem into two phases: First, we apply supervised learning for predicting the near future based on the present. We require that the predictor will be differentiable with respect to the representation of the present. Second, we model a full trajectory of the agent using a recurrent neural network, where unexplained factors are modeled as (additive) input nodes. This allows us to solve the long-term planning problem using supervised learning techniques and direct optimization over the recurrent neural network. Our approach enables us to learn robust policies by incorporating adversarial elements to the environment.