DOTmark - A Benchmark for Discrete Optimal Transport

TL;DR

The paper introduces DOTmark, a comprehensive benchmark dataset for discrete optimal transport, enabling systematic comparison of various algorithms on diverse image-based problems to advance computational methods.

Contribution

It provides a neutral, large-scale benchmark dataset for discrete optimal transport, along with a survey and performance evaluation of existing algorithms.

Findings

Traditional algorithms like transportation simplex perform well on small instances.

Recent approaches such as shielding neighborhood method show improved scalability.

Commercial solvers vary in performance depending on problem size.

Abstract

The Wasserstein metric or earth mover's distance (EMD) is a useful tool in statistics, machine learning and computer science with many applications to biological or medical imaging, among others. Especially in the light of increasingly complex data, the computation of these distances via optimal transport is often the limiting factor. Inspired by this challenge, a variety of new approaches to optimal transport has been proposed in recent years and along with these new methods comes the need for a meaningful comparison. In this paper, we introduce a benchmark for discrete optimal transport, called DOTmark, which is designed to serve as a neutral collection of problems, where discrete optimal transport methods can be tested, compared to one another, and brought to their limits on large-scale instances. It consists of a variety of grayscale images, in various resolutions and classes, such as several types of randomly generated images, classical test images and real data from microscopy. Along with the DOTmark we present a survey and a performance test for a cross section of established methods ranging from more traditional algorithms, such as the transportation simplex, to recently developed approaches, such as the shielding neighborhood method, and including also a comparison with commercial solvers.