GLEAN: Generative Learning for Eliminating Adversarial Noise

TL;DR

This paper introduces GLEAN, a generative learning approach using I2I networks to remove perturbations from digital art, enhancing defenses against style mimicry attacks in diffusion models.

Contribution

GLEAN is a novel method that improves upon existing protection tools like Glaze by effectively removing perturbations and addressing their limitations.

Findings

GLEAN successfully reduces perturbations in Glazed images.

GLEAN enhances the effectiveness of style mimicry attack prevention.

The method highlights limitations of current protection techniques.

Abstract

In the age of powerful diffusion models such as DALL-E and Stable Diffusion, many in the digital art community have suffered style mimicry attacks due to fine-tuning these models on their works. The ability to mimic an artist's style via text-to-image diffusion models raises serious ethical issues, especially without explicit consent. Glaze, a tool that applies various ranges of perturbations to digital art, has shown significant success in preventing style mimicry attacks, at the cost of artifacts ranging from imperceptible noise to severe quality degradation. The release of Glaze has sparked further discussions regarding the effectiveness of similar protection methods. In this paper, we propose GLEAN- applying I2I generative networks to strip perturbations from Glazed images, evaluating the performance of style mimicry attacks before and after GLEAN on the results of Glaze. GLEAN aims to support and enhance Glaze by highlighting its limitations and encouraging further development.