FAIRS -- Soft Focus Generator and Attention for Robust Object Segmentation from Extreme Points

TL;DR

FAIRS introduces a novel method for interactive object segmentation using extreme points and corrective clicks, employing a dual attention mechanism to improve accuracy and scalability across large datasets.

Contribution

It proposes an effective encoding of user inputs and a dual attention module, enabling the model to handle variable clicks and refine outputs for robust segmentation.

Findings

Significant improvement over state-of-the-art in dense object segmentation

Effective handling of variable number of user clicks including corrective clicks

High-quality training data generation demonstrated

Abstract

Semantic segmentation from user inputs has been actively studied to facilitate interactive segmentation for data annotation and other applications. Recent studies have shown that extreme points can be effectively used to encode user inputs. A heat map generated from the extreme points can be appended to the RGB image and input to the model for training. In this study, we present FAIRS -- a new approach to generate object segmentation from user inputs in the form of extreme points and corrective clicks. We propose a novel approach for effectively encoding the user input from extreme points and corrective clicks, in a novel and scalable manner that allows the network to work with a variable number of clicks, including corrective clicks for output refinement. We also integrate a dual attention module with our approach to increase the efficacy of the model in preferentially attending to the objects. We demonstrate that these additions help achieve significant improvements over state-of-the-art in dense object segmentation from user inputs, on multiple large-scale datasets. Through experiments, we demonstrate our method's ability to generate high-quality training data as well as its scalability in incorporating extreme points, guiding clicks, and corrective clicks in a principled manner.