Formulation and properties of a divergence used to compare probability measures without absolute continuity

TL;DR

This paper introduces a novel divergence measure for comparing probability distributions that does not require absolute continuity, extending the concepts of relative entropy and optimal transport, with applications in model uncertainty quantification.

Contribution

It proposes a new divergence that generalizes relative entropy, providing a representation involving optimal transport and entropy, and demonstrates its properties and computational aspects.

Findings

The divergence can compare measures without absolute continuity.

It has a representation as an infimum convolution of transport cost and entropy.

Properties useful for quantifying model uncertainty are established.

Abstract

This paper develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and exploit a representation as an infimum convolution of optimal transport cost and relative entropy. Also included are examples of computation and approximation of the divergence, and the demonstration of properties that are useful when one quantifies model uncertainty.