Scalable Sobolev IPM for Probability Measures on a Graph

TL;DR

This paper introduces a novel regularization technique for Sobolev IPM on graphs, enabling fast, closed-form computation and practical application in large-scale graph-based probability measure comparisons.

Contribution

It establishes a relation between Sobolev and weighted Lp norms, proposes a regularization for Sobolev IPM, and derives a closed-form expression leveraging graph structure for efficient computation.

Findings

Regularized Sobolev IPM allows fast computation on graphs.

The method enables practical applications in large-scale graph data.

Positive-definite kernels based on Sobolev IPM improve measure comparison.

Abstract

We investigate the Sobolev IPM problem for probability measures supported on a graph metric space. Sobolev IPM is an important instance of integral probability metrics (IPM), and is obtained by constraining a critic function within a unit ball defined by the Sobolev norm. In particular, it has been used to compare probability measures and is crucial for several theoretical works in machine learning. However, to our knowledge, there are no efficient algorithmic approaches to compute Sobolev IPM effectively, which hinders its practical applications. In this work, we establish a relation between Sobolev norm and weighted $L^p$-norm, and leverage it to propose a \emph{novel regularization} for Sobolev IPM. By exploiting the graph structure, we demonstrate that the regularized Sobolev IPM provides a \emph{closed-form} expression for fast computation. This advancement addresses long-standing computational challenges, and paves the way to apply Sobolev IPM for practical applications, even in large-scale settings. Additionally, the regularized Sobolev IPM is negative definite. Utilizing this property, we design positive-definite kernels upon the regularized Sobolev IPM, and provide preliminary evidences of their advantages for comparing probability measures on a given graph for document classification and topological data analysis.