AttriCtrl: Fine-Grained Control of Aesthetic Attribute Intensity in Diffusion Models

TL;DR

AttriCtrl introduces a lightweight, plug-and-play framework that enables precise, continuous control over multiple aesthetic attributes in diffusion models, enhancing personalization and diversity without retraining the entire model.

Contribution

It proposes a novel hybrid strategy for quantifying aesthetic attributes and a plug-and-play encoder for continuous control, addressing the limitations of existing discrete or global preference-based methods.

Findings

Achieves accurate, continuous control over multiple aesthetic attributes.

Enhances personalization and diversity in image generation.

Maintains model performance with negligible computational overhead.

Abstract

Diffusion models have recently become the dominant paradigm for image generation, yet existing systems struggle to interpret and follow numeric instructions for adjusting semantic attributes. In real-world creative scenarios, especially when precise control over aesthetic attributes is required, current methods fail to provide such controllability. This limitation partly arises from the subjective and context-dependent nature of aesthetic judgments, but more fundamentally stems from the fact that current text encoders are designed for discrete tokens rather than continuous values. Meanwhile, efforts on aesthetic alignment, often leveraging reinforcement learning, direct preference optimization, or architectural modifications, primarily align models with a global notion of human preference. While these approaches improve user experience, they overlook the multifaceted and compositional nature of aesthetics, underscoring the need for explicit disentanglement and independent control of aesthetic attributes. To address this gap, we introduce AttriCtrl, a lightweight framework for continuous aesthetic intensity control in diffusion models. It first defines relevant aesthetic attributes, then quantifies them through a hybrid strategy that maps both concrete and abstract dimensions onto a unified $[0,1]$ scale. A plug-and-play value encoder is then used to transform user-specified values into model-interpretable embeddings for controllable generation. Experiments show that AttriCtrl achieves accurate and continuous control over both single and multiple aesthetic attributes, significantly enhancing personalization and diversity. Crucially, it is implemented as a lightweight adapter while keeping the diffusion model frozen, ensuring seamless integration with existing frameworks such as ControlNet at negligible computational cost.