The Maximal Variation of Martingales of Probabilities and Repeated Games with Incomplete Information

TL;DR

This paper establishes tight bounds on the variation of probability martingales and applies these results to quantify the convergence rate of values in repeated games with incomplete information.

Contribution

It derives a sharp upper bound on the variation of probability martingales in terms of entropy and demonstrates its tightness, then applies this to bound the difference in game values.

Findings

Variation of martingales is bounded by rom entropy.

The bound or martingale variation is tight.

Game value differences decrease at a rate proportional to or large k.

Abstract

The variation of a martingale $p_0^k=p_0,...,p_k$ of probabilities on a finite (or countable) set $X$ is denoted $V(p_0^k)$ and defined by $V(p_0^k)=E(\sum_{t=1}^k|p_t-p_{t-1}|_1)$. It is shown that $V(p_0^k)\leq \sqrt{2kH(p_0)}$, where $H(p)$ is the entropy function $H(p)=-\sum_xp(x)\log p(x)$ and $\log$ stands for the natural logarithm. Therefore, if $d$ is the number of elements of $X$, then $V(p_0^k)\leq \sqrt{2k\log d}$. It is shown that the order of magnitude of the bound $\sqrt{2k\log d}$ is tight for $d\leq 2^k$: there is $C>0$ such that for every $k$ and $d\leq 2^k$ there is a martingale $p_0^k=p_0,...,p_k$ of probabilities on a set $X$ with $d$ elements, and with variation $V(p_0^k)\geq C\sqrt{2k\log d}$. An application of the first result to game theory is that the difference between $v_k$ and $\lim_kv_k$, where $v_k$ is the value of the $k$-stage repeated game with incomplete information on one side with $d$ states, is bounded by $|G|\sqrt{2k^{-1}\log d}$ (where $|G|$ is the maximal absolute value of a stage payoff). Furthermore, it is shown that the order of magnitude of this game theory bound is tight.