Learning Control for LQR with Unknown Packet Loss Rate Using Finite Channel Samples

TL;DR

This paper develops a method to learn control policies for LQR systems over unknown packet loss channels by estimating the loss rate from finite samples and analyzing the stability and sub-optimality of the resulting controller.

Contribution

It introduces a finite-sample-based approach to estimate packet loss rate and provides stability and sub-optimality guarantees for the resulting control policy.

Findings

Stability threshold for estimation error ensuring closed-loop stability.

Explicit bounds on sample complexity for stabilizing control.

Validation through numerical examples confirming theoretical results.

Abstract

This paper studies the linear quadratic regulator (LQR) problem over an unknown Bernoulli packet loss channel. The unknown loss rate is estimated using finite channel samples and a certainty-equivalence (CE) optimal controller is then designed by treating the estimate as the true rate. The stabilizing capability and sub-optimality of the CE controller critically depend on the estimation error of loss rate. For discrete-time linear systems, we provide a stability threshold for the estimation error to ensure closed-loop stability, and analytically quantify the sub-optimality in terms of the estimation error and the difference in modified Riccati equations. Next, we derive the upper bound on sample complexity for the CE controller to be stabilizing. Tailored results with less conservatism are delivered for scalar systems and n-dimensional systems with invertible input matrix. Moreover, we establish a sufficient condition, independent of the unknown loss rate, to verify whether the CE controller is stabilizing in a probabilistic sense. Finally, numerical examples are used to validate our results.