Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features

TL;DR

This paper introduces a weakly-supervised method for localizing manipulated regions in images by combining image-level detection networks with pre-trained segmentation models, eliminating the need for pixel-level annotations.

Contribution

It presents a novel approach that fuses multi-resolution features with segmentation maps for effective manipulation localization without pixel-wise labels.

Findings

Effective localization of manipulated regions achieved

No requirement for pixel-level annotations

Improved interpretability over existing methods

Abstract

The explosive growth of digital images and the widespread availability of image editing tools have made image manipulation detection an increasingly critical challenge. Current deep learning-based manipulation detection methods excel in achieving high image-level classification accuracy, they often fall short in terms of interpretability and localization of manipulated regions. Additionally, the absence of pixel-wise annotations in real-world scenarios limits the existing fully-supervised manipulation localization techniques. To address these challenges, we propose a novel weakly-supervised approach that integrates activation maps generated by image-level manipulation detection networks with segmentation maps from pre-trained models. Specifically, we build on our previous image-level work named WCBnet to produce multi-view feature maps which are subsequently fused for coarse localization. These coarse maps are then refined using detailed segmented regional information provided by pre-trained segmentation models (such as DeepLab, SegmentAnything and PSPnet), with Bayesian inference employed to enhance the manipulation localization. Experimental results demonstrate the effectiveness of our approach, highlighting the feasibility to localize image manipulations without relying on pixel-level labels.