Regularized Discrete Optimal Transport

TL;DR

This paper proposes a regularized and relaxed discrete optimal transport framework for image color manipulation, enabling robust color transfer, noise artifact removal, and color normalization across multiple images through convex optimization and barycenter computation.

Contribution

It introduces a novel regularized and relaxed formulation of discrete optimal transport tailored for image processing, including new algorithms for color transfer and barycenter computation.

Findings

Robust color transfer maps that handle large color palette variations.

Effective removal of colorization artifacts due to noise.

A convergent algorithm for computing barycenters for color normalization.

Abstract

This article introduces a generalization of the discrete optimal transport, with applications to color image manipulations. This new formulation includes a relaxation of the mass conservation constraint and a regularization term. These two features are crucial for image processing tasks, which necessitate to take into account families of multimodal histograms, with large mass variation across modes. The corresponding relaxed and regularized transportation problem is the solution of a convex optimization problem. Depending on the regularization used, this minimization can be solved using standard linear programming methods or first order proximal splitting schemes. The resulting transportation plan can be used as a color transfer map, which is robust to mass variation across images color palettes. Furthermore, the regularization of the transport plan helps to remove colorization artifacts due to noise amplification. We also extend this framework to the computation of barycenters of distributions. The barycenter is the solution of an optimization problem, which is separately convex with respect to the barycenter and the transportation plans, but not jointly convex. A block coordinate descent scheme converges to a stationary point of the energy. We show that the resulting algorithm can be used for color normalization across several images. The relaxed and regularized barycenter defines a common color palette for those images. Applying color transfer toward this average palette performs a color normalization of the input images.