CopyJudge: Automated Copyright Infringement Identification and Mitigation in Text-to-Image Diffusion Models

TL;DR

CopyJudge introduces an automated framework using vision-language models to identify and mitigate copyright infringement in AI-generated images, improving accuracy, interpretability, and generalization over existing methods.

Contribution

The paper presents a novel LVLM-based infringement detection and mitigation framework that simulates court-like judgments and optimizes prompts to reduce infringement in text-to-image diffusion models.

Findings

Achieves state-of-the-art infringement detection accuracy

Provides superior interpretability and generalization

Effectively mitigates memorization and IP infringement

Abstract

Assessing whether AI-generated images are substantially similar to source works is a crucial step in resolving copyright disputes. In this paper, we propose CopyJudge, a novel automated infringement identification framework that leverages large vision-language models (LVLMs) to simulate practical court processes for determining substantial similarity between copyrighted images and those generated by text-to-image diffusion models. Specifically, we employ an abstraction-filtration-comparison test framework based on the multi-LVLM debate to assess the likelihood of infringement and provide detailed judgment rationales. Based on these judgments, we further introduce a general LVLM-based mitigation strategy that automatically optimizes infringing prompts by avoiding sensitive expressions while preserving the non-infringing content. Furthermore, assuming the input noise is controllable, our approach can be enhanced by iteratively exploring non-infringing noise vectors within the diffusion latent space, even without modifying the original prompts. Experimental results show that our automated identification method achieves comparable state-of-the-art performance, while offering superior generalization and interpretability across various forms of infringement, and that our mitigation method more effectively mitigates memorization and IP infringement with a high degree of alignment to the original non-infringing expressions.