Youla-REN: Learning Nonlinear Feedback Policies with Robust Stability Guarantees

TL;DR

This paper introduces Youla-REN, a neural network-based nonlinear control framework with built-in stability guarantees, enabling unconstrained policy learning for uncertain systems.

Contribution

It develops a novel nonlinear Youla parameterization combined with recurrent equilibrium networks, ensuring globally exponentially stable policies with minimal assumptions.

Findings

Guarantees of stability for all policies in the search space

Simplifies learning with unconstrained optimization methods

Validated through diverse simulation examples

Abstract

This paper presents a parameterization of nonlinear controllers for uncertain systems building on a recently developed neural network architecture, called the recurrent equilibrium network (REN), and a nonlinear version of the Youla parameterization. The proposed framework has "built-in" guarantees of stability, i.e., all policies in the search space result in a contracting (globally exponentially stable) closed-loop system. Thus, it requires very mild assumptions on the choice of cost function and the stability property can be generalized to unseen data. Another useful feature of this approach is that policies are parameterized directly without any constraints, which simplifies learning by a broad range of policy-learning methods based on unconstrained optimization (e.g. stochastic gradient descent). We illustrate the proposed approach with a variety of simulation examples.