Lyapunov Neural ODE State-Feedback Control Policies

TL;DR

This paper introduces Lyapunov-NODE control (L-NODEC), a neural ODE-based method that learns stabilizing control policies with stability guarantees for constrained nonlinear systems, demonstrated through plasma medicine and dose delivery tasks.

Contribution

It presents a novel Lyapunov loss formulation for neural ODE control policies that ensures exponential stability and robustness in continuous-time optimal control problems.

Findings

L-NODEC guarantees exponential stability of the controlled system.

L-NODEC demonstrates robustness to initial state perturbations.

L-NODEC reduces inference time to reach target states.

Abstract

Deep neural networks are increasingly used as an effective parameterization of control policies in various learning-based control paradigms. For continuous-time optimal control problems (OCPs), which are central to many decision-making tasks, control policy learning can be cast as a neural ordinary differential equation (NODE) problem wherein state and control constraints are naturally accommodated. This paper presents a NODE approach to solving continuous-time OCPs for the case of stabilizing a known constrained nonlinear system around a target state. The approach, termed Lyapunov-NODE control (L-NODEC), uses a novel Lyapunov loss formulation that incorporates an exponentially-stabilizing control Lyapunov function to learn a state-feedback neural control policy, bridging the gap of solving continuous-time OCPs via NODEs with stability guarantees. The proposed Lyapunov loss allows L-NODEC to guarantee exponential stability of the controlled system, as well as its adversarial robustness to perturbations to the initial state. The performance of L-NODEC is illustrated in two problems, including a dose delivery problem in plasma medicine. In both cases, L-NODEC effectively stabilizes the controlled system around the target state despite perturbations to the initial state and reduces the inference time necessary to reach the target.