Judging from Support-set: A New Way to Utilize Few-Shot Segmentation for Segmentation Refinement Process

TL;DR

This paper introduces JFS, a novel method that uses a few-shot segmentation model to evaluate the success of segmentation refinement, improving reliability in image segmentation tasks.

Contribution

The paper proposes a new approach leveraging off-the-shelf FSS models to judge segmentation refinement success, addressing a gap in existing methods.

Findings

JFS effectively evaluates segmentation refinement quality.

Demonstrated on PASCAL dataset with SegGPT and SEPL.

Potential to enhance segmentation reliability in applications.

Abstract

Segmentation refinement aims to enhance the initial coarse masks generated by segmentation algorithms. The refined masks are expected to capture more details and better contours of the target objects. Research on segmentation refinement has developed as a response to the need for high-quality image segmentations. However, to our knowledge, no method has been developed that can determine the success of segmentation refinement. Such a method could ensure the reliability of segmentation in applications where the outcome of the segmentation is important and fosters innovation in image processing technologies. To address this research gap, we propose Judging From Support-set (JFS), a method to judge the success of segmentation refinement leveraging an off-the-shelf few-shot segmentation (FSS) model. The traditional goal of the problem in FSS is to find a target object in a query image utilizing target information given by a support set. However, we propose a novel application of the FSS model in our evaluation pipeline for segmentation refinement methods. Given a coarse mask as input, segmentation refinement methods produce a refined mask; these two masks become new support masks for the FSS model. The existing support mask then serves as the test set for the FSS model to evaluate the quality of the refined segmentation by the segmentation refinement methods. We demonstrate the effectiveness of our proposed JFS framework by evaluating the SAM Enhanced Pseudo-Labels (SEPL) using SegGPT as the choice of FSS model on the PASCAL dataset. The results showed that JFS has the potential to determine whether the segmentation refinement process is successful.