Decentralized linear quadratic systems with major and minor agents and non-Gaussian noise

TL;DR

This paper analyzes a decentralized linear quadratic control system with a major agent and multiple minor agents under non-Gaussian noise, characterizing optimal strategies and showing how to implement best linear controls.

Contribution

It provides a novel characterization of optimal control strategies in a decentralized setting with non-Gaussian noise, including the use of MMSE and LLMS estimates for control.

Findings

Major agent's control is a linear function of its MMSE estimate.

Minor agents' control involves the major agent’s MMSE estimate and a correction term.

Replacing MMSE with LLMS estimates yields the best linear control strategy.

Abstract

A decentralized linear quadratic system with a major agent and a collection of minor agents is considered. The major agent affects the minor agents, but not vice versa. The state of the major agent is observed by all agents. In addition, the minor agents have a noisy observation of their local state. The noise processes is \emph{not} assumed to be Gaussian. The structures of the optimal strategy and the best linear strategy are characterized. It is shown that major agent's optimal control action is a linear function of the major agent's MMSE (minimum mean squared error) estimate of the system state while the minor agent's optimal control action is a linear function of the major agent's MMSE estimate of the system state and a "correction term" which depends on the difference of the minor agent's MMSE estimate of its local state and the major agent's MMSE estimate of the minor agent's local state. Since the noise is non-Gaussian, the minor agent's MMSE estimate is a non-linear function of its observation. It is shown that replacing the minor agent's MMSE estimate by its LLMS (linear least mean square) estimate gives the best linear control strategy. The results are proved using a direct method based on conditional independence, common-information-based splitting of state and control actions, and simplifying the per-step cost based on conditional independence, orthogonality principle, and completion of squares.