Extremum Problems with Total Variation Distance and their Applications

TL;DR

This paper provides a comprehensive analysis of extremum problems involving total variation distance, deriving closed-form solutions and exploring applications in probability approximation, entropy optimization, and decision theory.

Contribution

It introduces a unified approach to solve extremum problems with total variation constraints, including explicit solutions and support partitioning, applicable to abstract and discrete spaces.

Findings

Closed-form solutions for extremum measures are derived.

Partitioning of support sets is characterized for extremum measures.

Applications include probability approximation and entropy optimization.

Abstract

The aim of this paper is to investigate extremum problems with pay-off being the total variational distance metric defined on the space of probability measures, subject to linear functional constraints on the space of probability measures, and vice-versa; that is, with the roles of total variational metric and linear functional interchanged. Utilizing concepts from signed measures, the extremum probability measures of such problems are obtained in closed form, by identifying the partition of the support set and the mass of these extremum measures on the partition. The results are derived for abstract spaces; specifically, complete separable metric spaces known as Polish spaces, while the high level ideas are also discussed for denumerable spaces endowed with the discrete topology. These extremum problems often arise in many areas, such as, approximating a family of probability distributions by a given probability distribution, maximizing or minimizing entropy subject to total variational distance metric constraints, quantifying uncertainty of probability distributions by total variational distance metric, stochastic minimax control, and in many problems of information, decision theory, and minimax theory.