JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimization

TL;DR

JANUS is a lightweight, distribution-based framework that effectively jailbreaks text-to-image models by optimizing prompt distributions, revealing vulnerabilities in current safety measures and outperforming existing methods.

Contribution

The paper introduces JANUS, a novel, efficient framework that formulates jailbreak as prompt distribution optimization, avoiding large-scale generators and improving success rates over prior approaches.

Findings

JANUS achieves a jailbreak success rate of 43.15% on Stable Diffusion 3.5.

JANUS outperforms state-of-the-art methods in success rate and safety filter bypass.

The approach exposes structural weaknesses in current T2I safety defenses.

Abstract

Text-to-image (T2I) models such as Stable Diffusion and DALLE remain susceptible to generating harmful or Not-Safe-For-Work (NSFW) content under jailbreak attacks despite deployed safety filters. Existing jailbreak attacks either rely on proxy-loss optimization instead of the true end-to-end objective, or depend on large-scale and costly RL-trained generators. Motivated by these limitations, we propose JANUS , a lightweight framework that formulates jailbreak as optimizing a structured prompt distribution under a black-box, end-to-end reward from the T2I system and its safety filters. JANUS replaces a high-capacity generator with a low-dimensional mixing policy over two semantically anchored prompt distributions, enabling efficient exploration while preserving the target semantics. On modern T2I models, we outperform state-of-the-art jailbreak methods, improving ASR-8 from 25.30% to 43.15% on Stable Diffusion 3.5 Large Turbo with consistently higher CLIP and NSFW scores. JANUS succeeds across both open-source and commercial models. These findings expose structural weaknesses in current T2I safety pipelines and motivate stronger, distribution-aware defenses. Warning: This paper contains model outputs that may be offensive.