A Bayesian Perspective on the Data-Driven LQR

TL;DR

This paper introduces a Bayesian approach to data-driven LQR that explicitly models uncertainty, leading to improved control performance especially with limited data.

Contribution

It develops a Bayesian formulation for both indirect and direct ddLQR, unifying them and enabling a tractable semidefinite program that accounts for model uncertainty.

Findings

Enhanced control performance in low-data regimes.

Decomposition of expected cost into certainty and variance terms.

Equivalent formulations for indirect and direct methods under the Bayesian perspective.

Abstract

The data-driven linear quadratic regulator (ddLQR) is a widely studied control method for unknown dynamical systems with disturbance. Existing approaches, both indirect, i.e., those that identify a model followed by model-based design, and direct, which bypasses the identification step, often rely on the certainty-equivalence principle and therefore do not explicitly account for model uncertainty. In this paper, we propose a Bayesian formulation for both indirect and direct ddLQR that incorporates posterior uncertainty into the control design. The resulting expected cost decomposes into a certainty-equivalence term and a variance-dependent term, providing a principled interpretation of regularization. We further show that the indirect and direct formulations are equivalent under this perspective. The resulting direct method admits a tractable semidefinite program whose size is independent of the data length. Numerical simulations demonstrate improved optimality gap and closed-loop stability, particularly in low-data regimes.