Concept Reachability in Diffusion Models: Beyond Dataset Constraints

TL;DR

This paper explores how steering intermediate activations in diffusion models can enable concept reachability beyond dataset limitations, revealing phase transitions and the importance of intervention timing for effective control.

Contribution

It introduces a new experimental framework to analyze concept reachability in latent space, highlighting phase transitions and the impact of intervention timing, offering practical control strategies.

Findings

Concept reachability exhibits a phase transition with few samples.

Intervention timing critically affects reachability.

Steering remains effective despite dataset quality decline.

Abstract

Despite significant advances in quality and complexity of the generations in text-to-image models, prompting does not always lead to the desired outputs. Controlling model behaviour by directly steering intermediate model activations has emerged as a viable alternative allowing to reach concepts in latent space that may otherwise remain inaccessible by prompt. In this work, we introduce a set of experiments to deepen our understanding of concept reachability. We design a training data setup with three key obstacles: scarcity of concepts, underspecification of concepts in the captions, and data biases with tied concepts. Our results show: (i) concept reachability in latent space exhibits a distinct phase transition, with only a small number of samples being sufficient to enable reachability, (ii) where in the latent space the intervention is performed critically impacts reachability, showing that certain concepts are reachable only at certain stages of transformation, and (iii) while prompting ability rapidly diminishes with a decrease in quality of the dataset, concepts often remain reliably reachable through steering. Model providers can leverage this to bypass costly retraining and dataset curation and instead innovate with user-facing control mechanisms.