Hierarchical image simplification and segmentation based on Mumford-Shah-salient level line selection

TL;DR

This paper introduces a hierarchical image simplification and segmentation method based on Mumford-Shah functional minimization, utilizing the tree of shapes to produce multiscale, salient structures with state-of-the-art results.

Contribution

It presents a novel approach that leverages the tree of shapes and energy minimization to generate hierarchical segmentations without relying on input hierarchies.

Findings

Achieves state-of-the-art segmentation performance.

Effectively identifies salient image structures.

Produces meaningful multiscale hierarchies.

Abstract

Hierarchies, such as the tree of shapes, are popular representations for image simplification and segmentation thanks to their multiscale structures. Selecting meaningful level lines (boundaries of shapes) yields to simplify image while preserving intact salient structures. Many image simplification and segmentation methods are driven by the optimization of an energy functional, for instance the celebrated Mumford-Shah functional. In this paper, we propose an efficient approach to hierarchical image simplification and segmentation based on the minimization of the piecewise-constant Mumford-Shah functional. This method conforms to the current trend that consists in producing hierarchical results rather than a unique partition. Contrary to classical approaches which compute optimal hierarchical segmentations from an input hierarchy of segmentations, we rely on the tree of shapes, a unique and well-defined representation equivalent to the image. Simply put, we compute for each level line of the image an attribute function that characterizes its persistence under the energy minimization. Then we stack the level lines from meaningless ones to salient ones through a saliency map based on extinction values defined on the tree-based shape space. Qualitative illustrations and quantitative evaluation on Weizmann segmentation evaluation database demonstrate the state-of-the-art performance of our method.