Masked Cross-image Encoding for Few-shot Segmentation

TL;DR

This paper introduces Masked Cross-image Encoding (MCE), a novel method for few-shot segmentation that captures inter-image dependencies and enhances feature interaction to improve segmentation accuracy on standard benchmarks.

Contribution

The paper proposes MCE, a joint learning approach that models mutual dependencies between support and query images, addressing limitations of prior class-wise descriptor methods in FSS.

Findings

MCE outperforms existing methods on PASCAL-5^i and COCO-20^i benchmarks.

MCE effectively captures inter-image dependencies, leading to better segmentation results.

The approach demonstrates strong meta-learning capabilities in few-shot segmentation tasks.

Abstract

Few-shot segmentation (FSS) is a dense prediction task that aims to infer the pixel-wise labels of unseen classes using only a limited number of annotated images. The key challenge in FSS is to classify the labels of query pixels using class prototypes learned from the few labeled support exemplars. Prior approaches to FSS have typically focused on learning class-wise descriptors independently from support images, thereby ignoring the rich contextual information and mutual dependencies among support-query features. To address this limitation, we propose a joint learning method termed Masked Cross-Image Encoding (MCE), which is designed to capture common visual properties that describe object details and to learn bidirectional inter-image dependencies that enhance feature interaction. MCE is more than a visual representation enrichment module; it also considers cross-image mutual dependencies and implicit guidance. Experiments on FSS benchmarks PASCAL-$5^i$ and COCO-$20^i$ demonstrate the advanced meta-learning ability of the proposed method.