Dense Affinity Matching for Few-Shot Segmentation

TL;DR

This paper introduces Dense Affinity Matching, a novel framework for Few-Shot Segmentation that leverages dense pixel-to-pixel and pixel-to-patch relations, improving performance across various benchmarks with high efficiency.

Contribution

The paper proposes a dense affinity matching framework with a hysteretic spatial filtering module for better support-query interaction in FSS, especially under cross-domain scenarios.

Findings

Outperforms existing methods on ten benchmarks.

Requires only 0.68M parameters, demonstrating efficiency.

Shows significant improvements in cross-domain FSS tasks.

Abstract

Few-Shot Segmentation (FSS) aims to segment the novel class images with a few annotated samples. In this paper, we propose a dense affinity matching (DAM) framework to exploit the support-query interaction by densely capturing both the pixel-to-pixel and pixel-to-patch relations in each support-query pair with the bidirectional 3D convolutions. Different from the existing methods that remove the support background, we design a hysteretic spatial filtering module (HSFM) to filter the background-related query features and retain the foreground-related query features with the assistance of the support background, which is beneficial for eliminating interference objects in the query background. We comprehensively evaluate our DAM on ten benchmarks under cross-category, cross-dataset, and cross-domain FSS tasks. Experimental results demonstrate that DAM performs very competitively under different settings with only 0.68M parameters, especially under cross-domain FSS tasks, showing its effectiveness and efficiency.