Target-aware Bi-Transformer for Few-shot Segmentation

TL;DR

The paper introduces TBTNet, a lightweight and efficient few-shot segmentation model that uses a target-aware transformer to focus on foreground features, achieving fast convergence and strong performance on standard benchmarks.

Contribution

It proposes a novel Target-aware Bi-Transformer Network with a dedicated transformer layer, reducing complexity and training time for few-shot segmentation.

Findings

Achieves state-of-the-art results on PASCAL-5i and COCO-20i benchmarks.

Model is the lightest with only 0.4M parameters.

Converges in 10-25% of the training epochs of traditional methods.

Abstract

Traditional semantic segmentation tasks require a large number of labels and are difficult to identify unlearned categories. Few-shot semantic segmentation (FSS) aims to use limited labeled support images to identify the segmentation of new classes of objects, which is very practical in the real world. Previous researches were primarily based on prototypes or correlations. Due to colors, textures, and styles are similar in the same image, we argue that the query image can be regarded as its own support image. In this paper, we proposed the Target-aware Bi-Transformer Network (TBTNet) to equivalent treat of support images and query image. A vigorous Target-aware Transformer Layer (TTL) also be designed to distill correlations and force the model to focus on foreground information. It treats the hypercorrelation as a feature, resulting a significant reduction in the number of feature channels. Benefit from this characteristic, our model is the lightest up to now with only 0.4M learnable parameters. Futhermore, TBTNet converges in only 10% to 25% of the training epochs compared to traditional methods. The excellent performance on standard FSS benchmarks of PASCAL-5i and COCO-20i proves the efficiency of our method. Extensive ablation studies were also carried out to evaluate the effectiveness of Bi-Transformer architecture and TTL.