Kernel Mean Embedding Topology: Weak and Strong Forms for Stochastic Kernels and Implications for Model Learning

TL;DR

This paper introduces a new topology for stochastic kernels called Kernel Mean Embedding Topology, which has weak and strong forms, enabling better analysis of model learning, robustness, and optimality in stochastic control.

Contribution

The paper defines a novel Kernel Mean Embedding Topology for stochastic kernels, connecting it with existing topologies and exploring its implications for model robustness and learning.

Findings

Both the w*-topology and kernel mean embedding topology are relatively compact but not closed.

The Young narrow topology is closed but not relatively compact.

The strong form of the topology aids in analyzing robustness and learning in stochastic models.

Abstract

We introduce a novel topology, called Kernel Mean Embedding Topology, for stochastic kernels, in a weak and strong form. This topology, defined on the spaces of Bochner integrable functions from a signal space to a space of probability measures endowed with a Hilbert space structure, allows for a versatile formulation. This construction allows one to obtain both a strong and weak formulation. (i) For its weak formulation, we highlight the utility on relaxed policy spaces, and investigate connections with the Young narrow topology and Borkar (or \( w^* \))-topology, and establish equivalence properties. We report that, while both the \( w^* \)-topology and kernel mean embedding topology are relatively compact, they are not closed. Conversely, while the Young narrow topology is closed, it lacks relative compactness. (ii) We show that the strong form provides an appropriate formulation for placing topologies on spaces of models characterized by stochastic kernels with explicit robustness and learning theoretic implications on optimal stochastic control under discounted or average cost criteria. (iii) We thus show that this topology possesses several properties making it ideal to study optimality and approximations (under the weak formulation) and robustness (under the strong formulation) for many applications.