A PAC-Bayes Approach for Controlling Unknown Linear Discrete-time Systems

TL;DR

This paper introduces a PAC-Bayes framework for designing controllers for unknown stochastic linear systems, providing performance guarantees and efficient algorithms that work for both finite and infinite controller spaces.

Contribution

It develops a novel PAC-Bayes-based approach with theoretical guarantees for controlling unknown linear systems, including bounds for unbounded quadratic costs.

Findings

Controllers achieve performance comparable to LQG in numerical tests.

The bound holds for unbounded quadratic cost functions.

Proposes efficient algorithms with theoretical guarantees.

Abstract

This paper presents a PAC-Bayes framework for learning controllers for unknown stochastic linear discrete-time systems, where the system parameters are drawn from a fixed but unknown distribution. We derive a data-dependent high probability bound on the performance of any learned (stochastic) controller, and propose novel efficient learning algorithms with theoretical guarantees, which can be implemented for both finite and infinite controller spaces. Compared to prior work, our bound holds for unbounded quadratic cost. In the special case where LQG is optimal, our numerical results suggest that the learned controllers achieve comparable performance to LQG.