Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation

TL;DR

This paper introduces a new dataset and a novel neural network architecture for object segmentation based on personal fixations, improving accuracy and boundary preservation in interactive image segmentation tasks.

Contribution

The study presents the first dedicated PFOS dataset and a new OLBP network with object localization and boundary preservation modules, advancing fixation-based segmentation methods.

Findings

OLBP outperforms 17 state-of-the-art methods.

The dataset facilitates further research in fixation-based segmentation.

OLBP effectively preserves object boundaries during segmentation.

Abstract

As a natural way for human-computer interaction, fixation provides a promising solution for interactive image segmentation. In this paper, we focus on Personal Fixations-based Object Segmentation (PFOS) to address issues in previous studies, such as the lack of appropriate dataset and the ambiguity in fixations-based interaction. In particular, we first construct a new PFOS dataset by carefully collecting pixel-level binary annotation data over an existing fixation prediction dataset, such dataset is expected to greatly facilitate the study along the line. Then, considering characteristics of personal fixations, we propose a novel network based on Object Localization and Boundary Preservation (OLBP) to segment the gazed objects. Specifically, the OLBP network utilizes an Object Localization Module (OLM) to analyze personal fixations and locates the gazed objects based on the interpretation. Then, a Boundary Preservation Module (BPM) is designed to introduce additional boundary information to guard the completeness of the gazed objects. Moreover, OLBP is organized in the mixed bottom-up and top-down manner with multiple types of deep supervision. Extensive experiments on the constructed PFOS dataset show the superiority of the proposed OLBP network over 17 state-of-the-art methods, and demonstrate the effectiveness of the proposed OLM and BPM components. The constructed PFOS dataset and the proposed OLBP network are available at https://github.com/MathLee/OLBPNet4PFOS.