Formal Entropy-Regularized Control of Stochastic Systems

TL;DR

This paper develops a formal framework for entropy-aware control of continuous stochastic systems by bounding entropy differences between continuous and discretized models, enabling predictable and performance-guaranteed control synthesis.

Contribution

It introduces bounds on entropy for discretized systems and extends formal abstraction techniques to entropy-based performance measures in continuous stochastic control.

Findings

Bounds on entropy of system discretizations established

Entropy-aware controllers can be synthesized with formal guarantees

Case studies demonstrate effectiveness of the approach

Abstract

Analyzing and controlling system entropy is a powerful tool for regulating predictability of control systems. Applications benefiting from such approaches range from reinforcement learning and data security to human-robot collaboration. In continuous-state stochastic systems, accurate entropy analysis and control remains a challenge. In recent years, finite-state abstractions of continuous systems have enabled control synthesis with formal performance guarantees on objectives such as stage costs. However, these results do not extend to entropy-based performance measures. We solve this problem by first obtaining bounds on the entropy of system discretizations using traditional formal-abstractions results, and then obtaining an additional bound on the difference between the entropy of a continuous distribution and that of its discretization. The resulting theory enables formal entropy-aware controller synthesis that trades predictability against control performance while preserving formal guarantees for the original continuous system. More specifically, we focus on minimizing the linear combination of the KL divergence of the system trajectory distribution to uniform -- our system entropy metric -- and a generic cumulative cost. We note that the bound we derive on the difference between the KL divergence to uniform of a given continuous distribution and its discretization can also be relevant in more general information-theoretic contexts. A set of case studies illustrates the effectiveness of the method.