Convex Analysis in Decentralized Stochastic Control, Strategic Measures and Optimal Solutions

TL;DR

This paper explores the convexity and structural properties of strategic measures in decentralized stochastic control, providing conditions for optimality and demonstrating that randomness does not enhance team performance.

Contribution

It introduces a convex analytical framework for decentralized control problems, characterizes the convexity of strategic measures, and identifies conditions for optimal policies in complex team settings.

Findings

Set of strategic measures is convex for classical teams.

Extreme points of relaxed sets are deterministic policies.

Randomness does not improve team performance in optimal control.

Abstract

This paper is concerned with the properties of the sets of strategic measures induced by admissible team policies in decentralized stochastic control and the convexity properties in dynamic team problems. To facilitate a convex analytical approach, strategic measures for team problems are introduced. Properties such as convexity, and compactness and Borel measurability under weak convergence topology are studied, and sufficient conditions for each of these properties are presented. These lead to existence of and structural results for optimal policies. It will be shown that the set of strategic measures for teams which are not classical is in general non-convex, but the extreme points of a relaxed set consist of deterministic team policies, which lead to their optimality for a given team problem under an expected cost criterion. Externally provided independent common randomness for static teams or private randomness for dynamic teams do not improve the team performance. The problem of when a sequential team problem is convex is studied and necessary and sufficient conditions for problems which include teams with a non-classical information structure are presented. Implications of this analysis in identifying probability and information structure dependent convexity properties are presented.