Dissipative Imitation Learning for Discrete Dynamic Output Feedback Control with Sparse Data Sets

TL;DR

This paper introduces a dissipative imitation learning approach for discrete dynamic output feedback control that guarantees stability with sparse data and minimal plant model knowledge, using iterative convex overbounding and projected gradient descent.

Contribution

It proposes a novel IO stability-based dissipative imitation learning method that ensures closed-loop stability with limited data and plant model information.

Findings

Achieves closed-loop stability with sparse data sets.

Successfully mimics expert controller behavior.

Outperforms traditional methods in stability and performance.

Abstract

Imitation learning enables the synthesis of controllers for complex objectives and highly uncertain plant models. However, methods to provide stability guarantees to imitation learned controllers often rely on large amounts of data and/or known plant models. In this paper, we explore an input-output (IO) stability approach to dissipative imitation learning, which achieves stability with sparse data sets and with little known about the plant model. A closed-loop stable dynamic output feedback controller is learned using expert data, a coarse IO plant model, and a new constraint to enforce dissipativity on the learned controller. While the learning objective is nonconvex, iterative convex overbounding (ICO) and projected gradient descent (PGD) are explored as methods to successfully learn the controller. This new imitation learning method is applied to two unknown plants and compared to traditionally learned dynamic output feedback controller and neural network controller. With little knowledge of the plant model and a small data set, the dissipativity constrained learned controller achieves closed loop stability and successfully mimics the behavior of the expert controller, while other methods often fail to maintain stability and achieve good performance.