Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models

TL;DR

This paper introduces the task of Modelship Attribution, aiming to trace multi-stage image manipulations by different generative models, and proposes a transformer-based method to identify and sequence these models' contributions.

Contribution

It is the first to systematically model multi-stage manipulation attribution and introduces a novel transformer framework, along with a new dataset for this complex real-world scenario.

Findings

The proposed method effectively attributes multi-stage manipulations.

The modelship dataset contains 83,700 images for training and evaluation.

Extensive experiments show superior performance over existing methods.

Abstract

As generative techniques become increasingly accessible, authentic visuals are frequently subjected to iterative alterations by various individuals employing a variety of tools. Currently, to avoid misinformation and ensure accountability, a lot of research on detection and attribution is emerging. Although these methods demonstrate promise in single-stage manipulation scenarios, they fall short when addressing complex real-world iterative manipulation. In this paper, we are the first, to the best of our knowledge, to systematically model this real-world challenge and introduce a novel method to solve it. We define a task called "Modelship Attribution", which aims to trace the evolution of manipulated images by identifying the generative models involved and reconstructing the sequence of edits they performed. To realistically simulate this scenario, we utilize three generative models, StyleMapGAN, DiffSwap, and FacePartsSwap, that sequentially modify distinct regions of the same image. This process leads to the creation of the first modelship dataset, comprising 83,700 images (16,740 images*5). Given that later edits often overwrite the fingerprints of earlier models, the focus shifts from extracting blended fingerprints to characterizing each model's distinctive editing patterns. To tackle this challenge, we introduce the modelship attribution transformer (MAT), a purpose-built framework designed to effectively recognize and attribute the contributions of various models within complex, multi-stage manipulation workflows. Through extensive experiments and comparative analysis with other related methods, our results, including comprehensive ablation studies, demonstrate that the proposed approach is a highly effective solution for modelship attribution.