Trimap-guided Feature Mining and Fusion Network for Natural Image Matting

TL;DR

This paper introduces a novel trimap-guided feature mining and fusion network for natural image matting, leveraging semantic guidance and multi-scale pooling to improve accuracy and efficiency, and provides a new dataset for high-quality matting.

Contribution

The paper proposes a trimap-guided network with novel modules for feature mining and fusion, achieving significant performance improvements and efficiency over existing methods.

Findings

Achieves 13% and 25% relative improvement on SAD on Composition-1k and CIOM datasets.

Outperforms state-of-the-art methods on multiple benchmarks.

Uses fewer parameters and FLOPs than strong baselines.

Abstract

Utilizing trimap guidance and fusing multi-level features are two important issues for trimap-based matting with pixel-level prediction. To utilize trimap guidance, most existing approaches simply concatenate trimaps and images together to feed a deep network or apply an extra network to extract more trimap guidance, which meets the conflict between efficiency and effectiveness. For emerging content-based feature fusion, most existing matting methods only focus on local features which lack the guidance of a global feature with strong semantic information related to the interesting object. In this paper, we propose a trimap-guided feature mining and fusion network consisting of our trimap-guided non-background multi-scale pooling (TMP) module and global-local context-aware fusion (GLF) modules. Considering that trimap provides strong semantic guidance, our TMP module focuses effective feature mining on interesting objects under the guidance of trimap without extra parameters. Furthermore, our GLF modules use global semantic information of interesting objects mined by our TMP module to guide an effective global-local context-aware multi-level feature fusion. In addition, we build a common interesting object matting (CIOM) dataset to advance high-quality image matting. Particularly, results on the Composition-1k and our CIOM show that our TMFNet achieves 13% and 25% relative improvement on SAD, respectively, against a strong baseline with fewer parameters and 14% fewer FLOPs. Experimental results on the Composition-1k test set, Alphamatting benchmark, and our CIOM test set demonstrate that our method outperforms state-of-the-art approaches. Our code and models are available at https://github.com/Serge-weihao/TMF-Matting.