Contractive Dynamical Imitation Policies for Efficient Out-of-Sample Recovery

TL;DR

This paper introduces a novel framework for imitation policies based on contractive dynamical systems, ensuring reliable out-of-sample recovery and convergence, with theoretical guarantees and empirical validation in robotics tasks.

Contribution

It presents a new approach using contractive dynamical systems and recurrent equilibrium networks to improve out-of-sample robustness in imitation learning.

Findings

Significant out-of-sample performance improvements in robotic tasks

Theoretical bounds on worst-case and expected loss established

Policy convergence guaranteed under all parameter choices

Abstract

Imitation learning is a data-driven approach to learning policies from expert behavior, but it is prone to unreliable outcomes in out-of-sample (OOS) regions. While previous research relying on stable dynamical systems guarantees convergence to a desired state, it often overlooks transient behavior. We propose a framework for learning policies modeled by contractive dynamical systems, ensuring that all policy rollouts converge regardless of perturbations, and in turn, enable efficient OOS recovery. By leveraging recurrent equilibrium networks and coupling layers, the policy structure guarantees contractivity for any parameter choice, which facilitates unconstrained optimization. We also provide theoretical upper bounds for worst-case and expected loss to rigorously establish the reliability of our method in deployment. Empirically, we demonstrate substantial OOS performance improvements for simulated robotic manipulation and navigation tasks.