Improved Few-shot Segmentation by Redefinition of the Roles of Multi-level CNN Features

TL;DR

This paper redefines the roles of multi-level CNN features in few-shot segmentation, swapping their traditional importance, and demonstrates that iterative application of this reinterpretation improves state-of-the-art results.

Contribution

It introduces a novel reinterpretation of multi-level CNN features in few-shot segmentation by swapping their roles, enabling iterative refinement and improved performance.

Findings

Achieved new state-of-the-art on COCO-20$^i$ and PASCAL-5$^i$ datasets.

Demonstrated effectiveness of iterative application of the redefined feature roles.

Improved segmentation accuracy in 1-shot and 5-shot settings.

Abstract

This study is concerned with few-shot segmentation, i.e., segmenting the region of an unseen object class in a query image, given support image(s) of its instances. The current methods rely on the pretrained CNN features of the support and query images. The key to good performance depends on the proper fusion of their mid-level and high-level features; the former contains shape-oriented information, while the latter has class-oriented information. Current state-of-the-art methods follow the approach of Tian et al., which gives the mid-level features the primary role and the high-level features the secondary role. In this paper, we reinterpret this widely employed approach by redifining the roles of the multi-level features; we swap the primary and secondary roles. Specifically, we regard that the current methods improve the initial estimate generated from the high-level features using the mid-level features. This reinterpretation suggests a new application of the current methods: to apply the same network multiple times to iteratively update the estimate of the object's region, starting from its initial estimate. Our experiments show that this method is effective and has updated the previous state-of-the-art on COCO-20$^i$ in the 1-shot and 5-shot settings and on PASCAL-5$^i$ in the 1-shot setting.