Imposing Robust Structured Control Constraint on Reinforcement Learning of Linear Quadratic Regulator

TL;DR

This paper introduces a robust structured reinforcement learning approach for linear quadratic regulators that ensures stability and robustness in systems with unknown dynamics, enabling distributed control design.

Contribution

It develops a novel data-driven framework, RSRL, integrating structural constraints into RL for LQR, with theoretical guarantees on stability and convergence.

Findings

Validated on a multi-agent network simulation with 6 agents.

Demonstrated robustness and stability in the control design.

Enabled distributed control with structural constraints.

Abstract

This paper discusses learning a structured feedback control to obtain sufficient robustness to exogenous inputs for linear dynamic systems with unknown state matrix. The structural constraint on the controller is necessary for many cyber-physical systems, and our approach presents a design for any generic structure, paving the way for distributed learning control. The ideas from reinforcement learning (RL) in conjunction with control-theoretic sufficient stability and performance guarantees are used to develop the methodology. First, a model-based framework is formulated using dynamic programming to embed the structural constraint in the linear quadratic regulator (LQR) setting along with sufficient robustness conditions. Thereafter, we translate these conditions to a data-driven learning-based framework - robust structured reinforcement learning (RSRL) that enjoys the control-theoretic guarantees on stability and convergence. We validate our theoretical results with a simulation on a multi-agent network with 6 agents.