cuRegOT: A GPU-Accelerated Solver for Entropic-Regularized Optimal Transport

TL;DR

cuRegOT is a GPU-accelerated solver for entropic-regularized optimal transport that combines novel algorithmic optimizations to significantly improve computational speed and efficiency.

Contribution

The paper introduces cuRegOT, a GPU solver with innovative strategies like amortized symbolic analysis and fused kernels, enhancing OT computation performance.

Findings

cuRegOT achieves substantial speedups over existing GPU solvers.

The method maintains theoretical convergence guarantees.

Extensive experiments validate its efficiency across benchmarks.

Abstract

Optimal transport (OT) has emerged as a fundamental tool in modern machine learning, yet its computational cost remains a significant bottleneck for large-scale applications. While harnessing the massive parallelism of modern GPU hardware is critical for efficiency, the de facto standard Sinkhorn algorithm, despite its ease of parallelization, often suffers from slow convergence in challenging problems. More recently, the sparse-plus-low-rank quasi-Newton method offers a balance between convergence rate and per-iteration complexity; however, its efficiency on GPUs is severely hindered by the serial nature of sparse matrix symbolic analysis and irregular memory access patterns. To bridge this gap, we present cuRegOT, a high-performance GPU solver tailored for entropic-regularized OT. We introduce a suite of algorithmic and architectural optimizations, including an amortized symbolic analysis strategy to mitigate CPU bottlenecks, an asynchronous Sinkhorn iterates generation mechanism, and a fused kernel for bandwidth-efficient gradient evaluation. These strategies are backed by rigorous theoretical guarantees ensuring algorithmic convergence. Extensive numerical experiments demonstrate that cuRegOT achieves significant speedups over state-of-the-art GPU-based solvers across a variety of benchmark tasks.