Online Learning to Transport via the Minimal Selection Principle

TL;DR

This paper introduces the Online Learning to Transport (OLT) problem, connecting online learning, optimal transport, and PDEs through the minimal selection principle, and proposes the MSoE algorithm for infinite-dimensional decision spaces.

Contribution

It extends online learning to infinite-dimensional probability measures using the minimal selection principle and develops the MSoE algorithm for practical resource allocation.

Findings

Minimizing transport cost over time yields optimal regret bounds.

The MSoE algorithm applies beyond convex settings, enhancing practical relevance.

The framework bridges online learning, optimal transport, and PDEs.

Abstract

Motivated by robust dynamic resource allocation in operations research, we study the \textit{Online Learning to Transport} (OLT) problem where the decision variable is a probability measure, an infinite-dimensional object. We draw connections between online learning, optimal transport, and partial differential equations through an insight called the minimal selection principle, originally studied in the Wasserstein gradient flow setting by \citet{Ambrosio_2005}. This allows us to extend the standard online learning framework to the infinite-dimensional setting seamlessly. Based on our framework, we derive a novel method called the \textit{minimal selection or exploration (MSoE) algorithm} to solve OLT problems using mean-field approximation and discretization techniques. In the displacement convex setting, the main theoretical message underpinning our approach is that minimizing transport cost over time (via the minimal selection principle) ensures optimal cumulative regret upper bounds. On the algorithmic side, our MSoE algorithm applies beyond the displacement convex setting, making the mathematical theory of optimal transport practically relevant to non-convex settings common in dynamic resource allocation.