Distribution Regression with Sliced Wasserstein Kernels

TL;DR

This paper introduces a novel distribution regression method using Sliced Wasserstein kernels, offering improved geometric representation of distributions with strong theoretical guarantees and promising experimental results.

Contribution

It proposes an OT-based distribution regression approach leveraging Sliced Wasserstein kernels, addressing limitations of MMD-based methods with proven consistency and risk bounds.

Findings

The proposed method is universally consistent.

It achieves favorable excess risk bounds.

Preliminary experiments show competitive performance.

Abstract

The problem of learning functions over spaces of probabilities - or distribution regression - is gaining significant interest in the machine learning community. A key challenge behind this problem is to identify a suitable representation capturing all relevant properties of the underlying functional mapping. A principled approach to distribution regression is provided by kernel mean embeddings, which lifts kernel-induced similarity on the input domain at the probability level. This strategy effectively tackles the two-stage sampling nature of the problem, enabling one to derive estimators with strong statistical guarantees, such as universal consistency and excess risk bounds. However, kernel mean embeddings implicitly hinge on the maximum mean discrepancy (MMD), a metric on probabilities, which may fail to capture key geometrical relations between distributions. In contrast, optimal transport (OT) metrics, are potentially more appealing. In this work, we propose an OT-based estimator for distribution regression. We build on the Sliced Wasserstein distance to obtain an OT-based representation. We study the theoretical properties of a kernel ridge regression estimator based on such representation, for which we prove universal consistency and excess risk bounds. Preliminary experiments complement our theoretical findings by showing the effectiveness of the proposed approach and compare it with MMD-based estimators.