Robust Decentralized Control of Coupled Systems via Risk Sensitive Control of Decoupled or Simple Models with Measure Change

TL;DR

This paper introduces a robust decentralized control framework for coupled stochastic systems by leveraging risk-sensitive control of simplified models, providing bounds and stability insights for complex multi-agent interactions.

Contribution

It develops a novel robustness approach using risk-sensitive control for decentralized systems, enabling tractable solutions and bounds for interacting agents with complex couplings.

Findings

Provides a bound on the cost function using risk-sensitive measures and relative entropy.

Shows stability of solutions under small interaction perturbations.

Applies to mean-field models with large numbers of agents.

Abstract

Decentralized stochastic control problems with local information involve problems where multiple agents and subsystems which are coupled via dynamics and/or cost are present. Typically, however, the dynamics of such couplings is complex and difficult to precisely model, leading to questions on robustness in control design. Additionally, when such a coupling can be modeled, the problem arrived at is typically challenging and non-convex, due to decentralization of information. In this paper, we develop a robustness framework for optimal decentralized control of interacting agents, where we show that a decentralized control problem with interacting agents can be robustly designed by considering a risk-sensitive version of non-interacting agents/particles. This leads to a tractable robust formulation where we give a bound on the value of the cost function in terms of the risk-sensitive cost function for the non-interacting case plus a term involving the ``strength" of the interaction as measured by relative entropy. We will build on Gaussian measure theory and an associated variational equality. A particular application includes mean-field models consisting of (a generally large number of) interacting agents which are often hard to solve for the case with small or moderate numbers of agents, leading to an interest in effective approximations and robustness. By adapting a risk-sensitivity parameter, we also robustly control a non-symmetrically interacting problem with mean-field cost by one which is symmetric with a risk-sensitive criterion, and in the limit of small interactions, show the stability of optimal solutions to perturbations.