Beyond Quadratic Costs in LQR: Bregman Divergence Control

TL;DR

This paper extends LQR control to a broader class of convex costs derived from Bregman divergence, enabling nonlinear feedback laws with stability guarantees for diverse control scenarios.

Contribution

It introduces a novel framework for control with Bregman divergence-based convex costs, generalizing quadratic cost methods to nonlinear, stable controllers.

Findings

Controllers are nonlinear and stable.

Applicable to safety, sparse, and bang-bang control.

Provides a full extension of quadratic LQR framework.

Abstract

In the past couple of decades, the use of ``non-quadratic" convex cost functions has revolutionized signal processing, machine learning, and statistics, allowing one to customize solutions to have desired structures and properties. However, the situation is not the same in control where the use of quadratic costs still dominates, ostensibly because determining the ``value function", i.e., the optimal expected cost-to-go, which is critical to the construction of the optimal controller, becomes computationally intractable as soon as one considers general convex costs. As a result, practitioners often resort to heuristics and approximations, such as model predictive control that only looks a few steps into the future. In the quadratic case, the value function is easily determined by solving Riccati equations. In this work, we consider a special class of convex cost functions constructed from Bregman divergence and show how, with appropriate choices, they can be used to fully extend the framework developed for the quadratic case. The resulting optimal controllers are infinite horizon, come with stability guarantees, and have state-feedback, or estimated state-feedback, laws. They exhibit a much wider range of behavior than their quadratic counterparts since the feedback laws are nonlinear. The approach can be applied to several cases of interest, including safety control, sparse control, and bang-bang control.