Safe Control and Learning Using Generalized Action Governor

TL;DR

This paper proposes the Generalized Action Governor (AG), a supervisory control scheme that enforces safety constraints in systems with uncertainties, enabling safe online learning and control in discrete-time systems.

Contribution

It develops a generalized AG theory for uncertain discrete-time systems, relaxing positive invariance to returnability, and introduces tailored AG designs and safe learning strategies.

Findings

AG guarantees constraint satisfaction during learning

Safe Q-learning and Koopman-based control are effectively integrated with AG

Numerical results demonstrate the effectiveness of the proposed methods

Abstract

This paper introduces the Generalized Action Governor (AG), a supervisory scheme that augments a nominal closed-loop system with the capability to enforce state and input constraints through online action adjustment. We develop a generalized AG theory for discrete-time systems under bounded uncertainties, and relax the usual requirement of positive invariance to returnability of a safe set. Based on the theory, we present tailored AG design procedures for linear systems and for discrete systems with finite state and action spaces. We further study safe online learning enabled by the AG and present two safe learning strategies, namely safe Q-learning and safe data-driven Koopman operator-based control, both integrated with the AG to guarantee constraint satisfaction during learning. Numerical results illustrate the proposed methods.