Demonstration-Efficient Guided Policy Search via Imitation of Robust Tube MPC

TL;DR

This paper introduces a demonstration-efficient imitation learning method that compresses a robust tube MPC into a neural network, enabling zero-shot transfer and improved robustness for quadrotor trajectory tracking.

Contribution

The paper presents a novel data augmentation technique using a Robust Tube MPC to enhance demonstration efficiency and robustness in imitation learning, facilitating zero-shot transfer from simulation to perturbed domains.

Findings

Outperforms DAgger and Domain Randomization in demonstration-efficiency

Enables zero-shot transfer from a single demonstration

Improves robustness to unseen perturbations

Abstract

We propose a demonstration-efficient strategy to compress a computationally expensive Model Predictive Controller (MPC) into a more computationally efficient representation based on a deep neural network and Imitation Learning (IL). By generating a Robust Tube variant (RTMPC) of the MPC and leveraging properties from the tube, we introduce a data augmentation method that enables high demonstration-efficiency, being capable to compensate the distribution shifts typically encountered in IL. Our approach opens the possibility of zero-shot transfer from a single demonstration collected in a nominal domain, such as a simulation or a robot in a lab/controlled environment, to a domain with bounded model errors/perturbations. Numerical and experimental evaluations performed on a trajectory tracking MPC for a quadrotor show that our method outperforms strategies commonly employed in IL, such as DAgger and Domain Randomization, in terms of demonstration-efficiency and robustness to perturbations unseen during training.