In the Shadows, Shape Priors Shine: Using Occlusion to Improve Multi-Region Segmentation

TL;DR

This paper introduces a novel multi-region segmentation algorithm that leverages shape priors and occlusion information, inspired by human visual perception, to improve accuracy in images with occluding objects.

Contribution

The proposed method uniquely combines shape priors with occlusion modeling, enabling better segmentation of partially occluded objects compared to existing approaches.

Findings

Significantly outperforms existing algorithms on natural and synthetic images.

Accurately recovers ground truth segmentation in synthetic datasets.

Effectively distinguishes shape deviations from occlusion effects.

Abstract

We present a new algorithm for multi-region segmentation of 2D images with objects that may partially occlude each other. Our algorithm is based on the observation hat human performance on this task is based both on prior knowledge about plausible shapes and taking into account the presence of occluding objects whose shape is already known - once an occluded region is identified, the shape prior can be used to guess the shape of the missing part. We capture the former aspect using a deep learning model of shape; for the latter, we simultaneously minimize the energy of all regions and consider only unoccluded pixels for data agreement. Existing algorithms incorporating object shape priors consider every object separately in turn and can't distinguish genuine deviation from the expected shape from parts missing due to occlusion. We show that our method significantly improves on the performance of a representative algorithm, as evaluated on both preprocessed natural and synthetic images. Furthermore, on the synthetic images, we recover the ground truth segmentation with good accuracy.