TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation Localization

TL;DR

TBNet is a novel two-stream boundary-aware neural network that effectively combines RGB and frequency information to improve the localization of manipulated regions in images, outperforming existing methods.

Contribution

The paper introduces TBNet, which jointly optimizes RGB, frequency, and boundary artifact features using adaptive modules for superior manipulation localization.

Findings

Outperforms state-of-the-art methods on four benchmarks.

Effectively integrates frequency and boundary information.

Achieves higher MCC and F1 scores.

Abstract

Finding tampered regions in images is a hot research topic in machine learning and computer vision. Although many image manipulation location algorithms have been proposed, most of them only focus on the RGB images with different color spaces, and the frequency information that contains the potential tampering clues is often ignored. In this work, a novel end-to-end two-stream boundary-aware network (abbreviated as TBNet) is proposed for generic image manipulation localization in which the RGB stream, the frequency stream, and the boundary artifact location are explored in a unified framework. Specifically, we first design an adaptive frequency selection module (AFS) to adaptively select the appropriate frequency to mine inconsistent statistics and eliminate the interference of redundant statistics. Then, an adaptive cross-attention fusion module (ACF) is proposed to adaptively fuse the RGB feature and the frequency feature. Finally, the boundary artifact location network (BAL) is designed to locate the boundary artifacts for which the parameters are jointly updated by the outputs of the ACF, and its results are further fed into the decoder. Thus, the parameters of the RGB stream, the frequency stream, and the boundary artifact location network are jointly optimized, and their latent complementary relationships are fully mined. The results of extensive experiments performed on four public benchmarks of the image manipulation localization task, namely, CASIA1.0, COVER, Carvalho, and In-The-Wild, demonstrate that the proposed TBNet can significantly outperform state-of-the-art generic image manipulation localization methods in terms of both MCC and F1.