ORES: Open-vocabulary Responsible Visual Synthesis

TL;DR

This paper introduces ORES, a framework enabling responsible visual synthesis by avoiding forbidden concepts through a two-stage intervention process involving large language models and diffusion models, with a new dataset and benchmarks.

Contribution

The paper formalizes the ORES task and proposes the TIN framework, integrating LLMs and diffusion models for concept avoidance in image synthesis.

Findings

Effective in reducing risks of image generation with forbidden concepts

Demonstrates the potential of LLMs in responsible visual synthesis

Provides a new dataset and benchmarks for the task

Abstract

Avoiding synthesizing specific visual concepts is an essential challenge in responsible visual synthesis. However, the visual concept that needs to be avoided for responsible visual synthesis tends to be diverse, depending on the region, context, and usage scenarios. In this work, we formalize a new task, Open-vocabulary Responsible Visual Synthesis (ORES), where the synthesis model is able to avoid forbidden visual concepts while allowing users to input any desired content. To address this problem, we present a Two-stage Intervention (TIN) framework. By introducing 1) rewriting with learnable instruction through a large-scale language model (LLM) and 2) synthesizing with prompt intervention on a diffusion synthesis model, it can effectively synthesize images avoiding any concepts but following the user's query as much as possible. To evaluate on ORES, we provide a publicly available dataset, baseline models, and benchmark. Experimental results demonstrate the effectiveness of our method in reducing risks of image generation. Our work highlights the potential of LLMs in responsible visual synthesis. Our code and dataset is public available.