A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with Applications in Partially Observable Markov Decision Processes

TL;DR

This paper extends Fenchel-Moreau-Rockafellar duality to Wasserstein-1 space, providing new tools for convex analysis and applications in POMDPs, including dual transportation inequalities and value function approximation.

Contribution

It introduces a dual representation of convex functionals on Wasserstein-1 space, extending duality results to Polish metric spaces and applying them to POMDPs.

Findings

Derived dual transportation inequalities.

Provided a dual expression for the Donsker-Varadhan formula.

Applied the theorem to approximate POMDP value functions.

Abstract

By using the fact that the space of all probability measures with finite support can be somehow completed in two different fashions, one generating the Arens-Eells space and another generating the Kantorovich-Wasserstein (Wasserstein-1) space, and by exploiting the duality relationship between the Arens-Eells space with the space of Lipschitz functions, we provide a dual representation of Fenchel-Moreau-Rockafellar type for proper convex functionals on Wasserstein-1. We retrieve dual transportation inequalities as a Corollary and we provide examples where the theorem can be used to easily prove dual expressions like the celebrated Donsker-Varadhan variational formula. Finally our result allows to write convex functions as the supremum over all linear functions that are generated by roots of its conjugate dual, something that we apply to the field of Partially observable Markov decision processes (POMDPs) to approximate the value function of a given POMDP by iterating level sets. This extends the method used in Smallwood 1973 for finite state spaces to the case were the state space is a Polish metric space.