React to Surprises: Stable-by-Design Neural Feedback Control and the Youla-REN

TL;DR

This paper introduces a neural feedback control parameterization based on a nonlinear Youla-Kucera approach combined with robust neural networks, ensuring stability during learning for complex nonlinear systems with partial observations.

Contribution

It proposes a novel unconstrained parameterization for stabilizing nonlinear policies that guarantees closed-loop stability by design, applicable to systems with partial observations and incremental stability requirements.

Findings

The parameterization guarantees stability with any two of the three difficulties: nonlinear dynamics, partial observation, stability requirements.

It maintains a weaker stability condition called d-tube contraction when all three difficulties are present.

Numerical experiments demonstrate effective learning of controllers with built-in stability for uncertain and complex systems.

Abstract

We study parameterizations of stabilizing nonlinear policies for learning-based control. We propose a structure based on a nonlinear version of the Youla-Kucera parameterization combined with robust neural networks such as the recurrent equilibrium network (REN). The resulting parameterizations are unconstrained, and hence can be searched over with first-order optimization methods, while always ensuring closed-loop stability by construction. We study the combination of (a) nonlinear dynamics, (b) partial observation, and (c) incremental closed-loop stability requirements (contraction and Lipschitzness). We find that with any two of these three difficulties, a contracting and Lipschitz Youla parameter always leads to contracting and Lipschitz closed loops. However, if all three hold, then incremental stability can be lost with exogenous disturbances. Instead, a weaker condition is maintained, which we call d-tube contraction and Lipschitzness. We further obtain converse results showing that the proposed parameterization covers all contracting and Lipschitz closed loops for certain classes of nonlinear systems. Numerical experiments illustrate the utility of our parameterization when learning controllers with built-in stability certificates for: (i) "economic" rewards without stabilizing effects; (ii) short training horizons; and (iii) uncertain systems.