Non-Sequential Decentralized Stochastic Control Revisited: Causality and Static Reducibility

TL;DR

This paper revisits causality in non-sequential decentralized stochastic control, establishing conditions under which such systems can be reduced to static models, enabling the application of sequential control techniques.

Contribution

It provides an alternative causality representation using imaginary agents and proves equivalence to causal implementability, extending static reduction methods to non-sequential systems.

Findings

Causality is equivalent to causal implementability and dead-lock freeness.

Under an absolute continuity condition, non-sequential systems can be reduced to static models.

Static reduction enables applying sequential control analysis to non-sequential systems.

Abstract

In decentralized stochastic control (or stochastic team theory) and game theory, if there is a pre-defined order in a system in which agents act, the system is called \textit{sequential}, otherwise it is non-sequential. Much of the literature on stochastic control theory, such as studies on the existence analysis, approximation methods, and on dynamic programming or other analytical or learning theoretic methods, have focused on sequential systems. Many complex practical systems, however, are non-sequential where the order of agents acting is random, and dependent on the realization of solution paths and prior actions taken. The study of such systems is particularly challenging as tools applicable for sequential models are not directly applicable. In this paper, we will first revisit the notion of Causality (a definition due to Witsenhausen and which has been refined by Andersland and Tekenetzis), and provide an alternative representation using imaginary agents. We show that Causality is equivalent to Causal Implementability (and Dead-Lock Freeness), thus, generalizing previous results. We show that Causality, under an absolute continuity condition, allows for an equivalent static model whose reduction is policy-independent. Since the static reduction method for sequential control problems (via change of measures or other techniques), has been shown to be very effective in arriving at existence, structural, approximation and learning theoretic results, our analysis facilitates much of the stochastic analysis available for sequential systems to also be applicable for a class of non-sequential systems.