Markov chain entropy games and the geometry of their Nash equilibria

TL;DR

This paper introduces a zero-sum game involving Markov generators and $f$-divergences, proving the existence of mixed strategy Nash equilibria and developing a method to compute them.

Contribution

It extends minimax results on $f$-divergences to Markov generator contexts and provides a computational approach for equilibria.

Findings

Existence of mixed strategy Nash equilibria in the game.

Development of a convergent projected subgradient method.

Connections to information centroids and Bayes risk.

Abstract

We introduce and study a two-player zero-sum game between a probabilist and Nature defined by a convex function $f$, a finite collection $\mathcal{B}$ of Markov generators (or its convex hull), and a target distribution $\pi$. The probabilist selects a mixed strategy $\mu \in \mathcal{P}(\mathcal{B})$, the set of probability measures on $\mathcal{B}$, while Nature adopts a pure strategy and selects a $\pi$-reversible Markov generator $M$. The probabilist receives a payoff equal to the $f$-divergence $D_f(M \| L)$, where $L$ is drawn according to $\mu$. We prove that this game always admits a mixed strategy Nash equilibrium and satisfies a minimax identity. In contrast, a pure strategy equilibrium may fail to exist. We develop a projected subgradient method to compute approximate mixed strategy equilibria with provable convergence guarantees. Connections to information centroids, Chebyshev centers, and Bayes risk are discussed. This paper extends earlier minimax results on $f$-divergences to the context of Markov generators.