RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model

TL;DR

This paper introduces RLCP, a reinforcement learning-based method that reduces copyright infringement in text-to-image diffusion models by optimizing a novel legal-grounded copyright metric, ensuring high-quality outputs.

Contribution

It proposes a novel RL-based framework with a copyright metric and regularization to effectively minimize infringement while preserving image quality in diffusion models.

Findings

Significantly reduces copyright infringement risk.

Maintains high image quality during fine-tuning.

Effective across multiple datasets with mixed copyright content.

Abstract

The increasing sophistication of text-to-image generative models has led to complex challenges in defining and enforcing copyright infringement criteria and protection. Existing methods, such as watermarking and dataset deduplication, fail to provide comprehensive solutions due to the lack of standardized metrics and the inherent complexity of addressing copyright infringement in diffusion models. To deal with these challenges, we propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model, which minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset. Our approach begins with the introduction of a novel copyright metric grounded in copyright law and court precedents on infringement. We then utilize the Denoising Diffusion Policy Optimization (DDPO) framework to guide the model through a multi-step decision-making process, optimizing it using a reward function that incorporates our proposed copyright metric. Additionally, we employ KL divergence as a regularization term to mitigate some failure modes and stabilize RL fine-tuning. Experiments conducted on 3 mixed datasets of copyright and non-copyright images demonstrate that our approach significantly reduces copyright infringement risk while maintaining image quality.