Situational Perception Guided Image Matting

TL;DR

This paper introduces a novel image matting approach that leverages situational perception and visual-to-textual guidance to improve saliency detection and reduce annotation bias, outperforming existing methods.

Contribution

The proposed SPG-IM method integrates a Semantic Transformation module and an Adaptive Focal Transformation network to enhance global and local saliency in image matting.

Findings

Outperforms state-of-the-art image matting methods.

Effectively captures inter-object and environment saliency.

Reduces subjective bias in matting annotations.

Abstract

Most automatic matting methods try to separate the salient foreground from the background. However, the insufficient quantity and subjective bias of the current existing matting datasets make it difficult to fully explore the semantic association between object-to-object and object-to-environment in a given image. In this paper, we propose a Situational Perception Guided Image Matting (SPG-IM) method that mitigates subjective bias of matting annotations and captures sufficient situational perception information for better global saliency distilled from the visual-to-textual task. SPG-IM can better associate inter-objects and object-to-environment saliency, and compensate the subjective nature of image matting and its expensive annotation. We also introduce a textual Semantic Transformation (TST) module that can effectively transform and integrate the semantic feature stream to guide the visual representations. In addition, an Adaptive Focal Transformation (AFT) Refinement Network is proposed to adaptively switch multi-scale receptive fields and focal points to enhance both global and local details. Extensive experiments demonstrate the effectiveness of situational perception guidance from the visual-to-textual tasks on image matting, and our model outperforms the state-of-the-art methods. We also analyze the significance of different components in our model. The code will be released soon.