MPC-Inspired Neural Network Policies for Sequential Decision Making

TL;DR

This paper explores MPC-inspired neural network policies for sequential decision making, introducing an extension to DAgger for improved training, and demonstrating enhanced robustness and scalability in complex continuous environments.

Contribution

It presents a novel extension to the DAgger algorithm tailored for MPC-inspired neural policies, enabling scalable training of complex planning architectures.

Findings

MPC-inspired recurrent policies exhibit better robustness to disturbances.

The extended DAgger algorithm improves training performance and generalization.

Neural policies with planning structures outperform simpler models in continuous spaces.

Abstract

In this paper we investigate the use of MPC-inspired neural network policies for sequential decision making. We introduce an extension to the DAgger algorithm for training such policies and show how they have improved training performance and generalization capabilities. We take advantage of this extension to show scalable and efficient training of complex planning policy architectures in continuous state and action spaces. We provide an extensive comparison of neural network policies by considering feed forward policies, recurrent policies, and recurrent policies with planning structure inspired by the Path Integral control framework. Our results suggest that MPC-type recurrent policies have better robustness to disturbances and modeling error.