A Bottom-Up Approach to Class-Agnostic Image Segmentation

TL;DR

This paper introduces a novel bottom-up, class-agnostic image segmentation method that directly learns feature representations and clusters them for segmentation, demonstrating strong generalization and effectiveness on various datasets.

Contribution

The work presents a new bottom-up formulation for class-agnostic segmentation, using feature space supervision and mean-shift clustering, differing from traditional top-down approaches.

Findings

Exhibits strong generalization to class-based datasets

Effective in cell and nucleus segmentation tasks

Outperforms some existing methods in accuracy

Abstract

Class-agnostic image segmentation is a crucial component in automating image editing workflows, especially in contexts where object selection traditionally involves interactive tools. Existing methods in the literature often adhere to top-down formulations, following the paradigm of class-based approaches, where object detection precedes per-object segmentation. In this work, we present a novel bottom-up formulation for addressing the class-agnostic segmentation problem. We supervise our network directly on the projective sphere of its feature space, employing losses inspired by metric learning literature as well as losses defined in a novel segmentation-space representation. The segmentation results are obtained through a straightforward mean-shift clustering of the estimated features. Our bottom-up formulation exhibits exceptional generalization capability, even when trained on datasets designed for class-based segmentation. We further showcase the effectiveness of our generic approach by addressing the challenging task of cell and nucleus segmentation. We believe that our bottom-up formulation will offer valuable insights into diverse segmentation challenges in the literature.