Globally Optimal Joint Image Segmentation and Shape Matching Based on Wasserstein Modes

TL;DR

This paper introduces a novel variational framework for joint image segmentation and shape matching using Wasserstein modes, enabling globally optimal solutions with efficient optimization schemes.

Contribution

It develops a new functional based on optimal transport that unifies shape modeling and segmentation, with methods for both local and global optimization.

Findings

The approach achieves geometric invariance and models statistical shape variations.

The combined optimization schemes effectively avoid initialization issues.

Numerical experiments demonstrate robustness and flexibility across data types.

Abstract

A functional for joint variational object segmentation and shape matching is developed. The formulation is based on optimal transport w.r.t. geometric distance and local feature similarity. Geometric invariance and modelling of object-typical statistical variations is achieved by introducing degrees of freedom that describe transformations and deformations of the shape template. The shape model is mathematically equivalent to contour-based approaches but inference can be performed without conversion between the contour and region representations, allowing combination with other convex segmentation approaches and simplifying optimization. While the overall functional is non-convex, non-convexity is confined to a low-dimensional variable. We propose a locally optimal alternating optimization scheme and a globally optimal branch and bound scheme, based on adaptive convex relaxation. Combining both methods allows to eliminate the delicate initialization problem inherent to many contour based approaches while remaining computationally practical. The properties of the functional, its ability to adapt to a wide range of input data structures and the different optimization schemes are illustrated and compared by numerical experiments.