Genetic column generation: Fast computation of high-dimensional multi-marginal optimal transport problems

TL;DR

This paper presents a novel, efficient method for solving high-dimensional multi-marginal optimal transport problems using a combination of column generation, genetic learning, and insights from machine learning, achieving exact solutions on benchmarks.

Contribution

Introduces a new genetic column generation method for MMOT that overcomes computational bottlenecks and scales polynomially with problem size, a novel approach in this context.

Findings

Always finds exact optimizers on benchmark problems

Number of computational steps scales polynomially with problem size

Method outperforms traditional approaches in efficiency and accuracy

Abstract

We introduce a simple, accurate, and extremely efficient method for numerically solving the multi-marginal optimal transport (MMOT) problems arising in density functional theory. The method relies on (i) the sparsity of optimal plans [for $N$ marginals discretized by $\ell$ gridpoints each, general Kantorovich plans require $\ell^N$ gridpoints but the support of optimizers is of size $O(\ell\cdot N)$ [FV18]], (ii) the method of column generation (CG) from discrete optimization which to our knowledge has not hitherto been used in MMOT, and (iii) ideas from machine learning. The well-known bottleneck in CG consists in generating new candidate columns efficiently; we prove that in our context, finding the best new column is an NP-complete problem. To overcome this bottleneck we use a genetic learning method tailormade for MMOT in which the dual state within CG plays the role of an "adversary", in loose similarity to Wasserstein GANs. On a sequence of benchmark problems with up to 120 gridpoints and up to 30 marginals, our method always found the exact optimizers. Moreover, empirically the number of computational steps needed to find them appears to scale only polynomially when both $N$ and $\ell$ are simultaneously increased (while keeping their ratio fixed to mimic a thermodynamic limit of the particle system).