Coherent distributions: Hilbert space approach and duality

TL;DR

This paper develops a Hilbert space framework to analyze coherent distributions, providing tight bounds on expectations and covariances of related random variables, with broad applicability to quadratic and other functions.

Contribution

It introduces a Hilbert space approach to characterize and bound coherent distributions, extending results to covariance and general expectations using linear programming duality.

Findings

Established an upper bound for quadratic functions of coherent distributions.

Characterized functions for which the bounds are tight.

Provided tight bounds on covariance for specific and general success probabilities.

Abstract

Let $X$ be a Bernoulli random variable with the success probability $p$. We are interested in tight bounds on $\mathbb{E}[f(X_1,X_2)]$, where $X_i=\mathbb{E}[X| \mathcal{F}_i]$ and $\mathcal{F}_i$ are some sigma-algebras. This problem is closely related to understanding extreme points of the set of coherent distributions. A distribution on $[0,1]^2$ is called $\textit{coherent}$ if it can be obtained as the joint distribution of $(X_1, X_2)$ for some choice of $\mathcal{F}_i$. By treating random variables as vectors in a Hilbert space, we establish an upper bound for quadratic $f$, characterize $f$ for which this bound is tight, and show that such $f$ result in exposed coherent distributions with arbitrarily large support. As a corollary, we get a tight bound on $\mathrm{cov}\,(X_1,X_2)$ for $p\in [1/3,\,2/3]$. To obtain a tight bound on $\mathrm{cov}\,(X_1,X_2)$ for all $p$, we develop an approach based on linear programming duality. Its generality is illustrated by tight bounds on $\mathbb{E}[|X_1-X_2|^\alpha]$ for any $\alpha>0$ and $p=1/2$.