Learning to segment from object sizes

TL;DR

This paper introduces a novel training method for deep segmentation that leverages easily obtainable object size annotations instead of pixel-wise labels, reducing annotation effort while improving segmentation accuracy.

Contribution

It proposes a new algorithm that trains segmentation networks using size annotations, overcoming the challenge of non-differentiable size-based loss functions.

Findings

Improved segmentation performance with fewer pixel-wise annotations.

Effective training using datasets with many size annotations and few pixel-wise labels.

Demonstrated robustness of the approach across different datasets.

Abstract

Deep learning has proved particularly useful for semantic segmentation, a fundamental image analysis task. However, the standard deep learning methods need many training images with ground-truth pixel-wise annotations, which are usually laborious to obtain and, in some cases (e.g., medical images), require domain expertise. Therefore, instead of pixel-wise annotations, we focus on image annotations that are significantly easier to acquire but still informative, namely the size of foreground objects. We define the object size as the maximum Chebyshev distance between a foreground and the nearest background pixel. We propose an algorithm for training a deep segmentation network from a dataset of a few pixel-wise annotated images and many images with known object sizes. The algorithm minimizes a discrete (non-differentiable) loss function defined over the object sizes by sampling the gradient and then using the standard back-propagation algorithm. Experiments show that the new approach improves the segmentation performance.