Numerical Methods for Large-Scale Optimal Transport

TL;DR

This paper reviews advanced numerical methods for solving large-scale optimal transport problems, emphasizing entropic regularization, Sinkhorn's algorithm, and primal-dual approaches, with applications to Wasserstein barycenters and distributed algorithms.

Contribution

It provides a comprehensive overview of modern algorithms for large-scale OT, highlighting recent developments and their applications in machine learning.

Findings

Sinkhorn's algorithm efficiently solves regularized OT problems.

Primal-dual methods offer flexible regularization options.

Distributed algorithms enable scalable Wasserstein barycenter computation.

Abstract

The optimal transport (OT) problem is a classical optimization problem having the form of linear programming. Machine learning applications put forward new computational challenges in its solution. In particular, the OT problem defines a distance between real-world objects such as images, videos, texts, etc., modeled as probability distributions. In this case, the large dimension of the corresponding optimization problem does not allow applying classical methods such as network simplex or interior-point methods. This challenge was overcome by introducing entropic regularization and using the efficient Sinkhorn's algorithm to solve the regularized problem. A flexible alternative is the accelerated primal-dual gradient method, which can use any strongly-convex regularization. We discuss these algorithms and other related problems such as approximating the Wasserstein barycenter together with efficient algorithms for its solution, including decentralized distributed algorithms.