Image Co-segmentation via Multi-scale Local Shape Transfer

TL;DR

This paper introduces a novel multi-scale local shape transfer method for image co-segmentation, effectively handling appearance variations by transferring consistent local object shapes across images.

Contribution

It proposes a new approach that transfers patch-level local shapes using a multi-scale neighborhood system and locally linear embedding, improving segmentation robustness.

Findings

Performs comparably or better than state-of-the-art methods on standard datasets.

Achieves significant improvements on the challenging Fashionista dataset.

Robustly segments common objects despite appearance variations.

Abstract

Image co-segmentation is a challenging task in computer vision that aims to segment all pixels of the objects from a predefined semantic category. In real-world cases, however, common foreground objects often vary greatly in appearance, making their global shapes highly inconsistent across images and difficult to be segmented. To address this problem, this paper proposes a novel co-segmentation approach that transfers patch-level local object shapes which appear more consistent across different images. In our framework, a multi-scale patch neighbourhood system is first generated using proposal flow on arbitrary image-pair, which is further refined by Locally Linear Embedding. Based on the patch relationships, we propose an efficient algorithm to jointly segment the objects in each image while transferring their local shapes across different images. Extensive experiments demonstrate that the proposed method can robustly and effectively segment common objects from an image set. On iCoseg, MSRC and Coseg-Rep dataset, the proposed approach performs comparable or better than the state-of-thearts, while on a more challenging benchmark Fashionista dataset, our method achieves significant improvements.