Contrastive Enhancement Using Latent Prototype for Few-Shot Segmentation

TL;DR

This paper introduces a contrastive enhancement method with latent prototypes for few-shot segmentation, improving the utilization of support-query similarity and significantly boosting performance over state-of-the-art methods.

Contribution

It proposes a novel latent prototype sampling and contrastive enhancement modules that can be integrated into existing models to improve few-shot segmentation accuracy.

Findings

Achieves 5.9% and 7.3% improvements on Pascal-5^i and COCO-20^i datasets.

Effectively leverages latent class information to enhance support-query similarity.

Outperforms baseline methods in 1-shot and 5-shot segmentation tasks.

Abstract

Few-shot segmentation enables the model to recognize unseen classes with few annotated examples. Most existing methods adopt prototype learning architecture, where support prototype vectors are expanded and concatenated with query features to perform conditional segmentation. However, such framework potentially focuses more on query features while may neglect the similarity between support and query features. This paper proposes a contrastive enhancement approach using latent prototypes to leverage latent classes and raise the utilization of similarity information between prototype and query features. Specifically, a latent prototype sampling module is proposed to generate pseudo-mask and novel prototypes based on features similarity. The module conveniently conducts end-to-end learning and has no strong dependence on clustering numbers like cluster-based method. Besides, a contrastive enhancement module is developed to drive models to provide different predictions with the same query features. Our method can be used as an auxiliary module to flexibly integrate into other baselines for a better segmentation performance. Extensive experiments show our approach remarkably improves the performance of state-of-the-art methods for 1-shot and 5-shot segmentation, especially outperforming baseline by 5.9% and 7.3% for 5-shot task on Pascal-5^i and COCO-20^i. Source code is available at https://github.com/zhaoxiaoyu1995/CELP-Pytorch