ATHENA: Adaptive Test-Time Steering for Improving Count Fidelity in Diffusion Models

TL;DR

ATHENA is a test-time adaptive steering method that enhances object count accuracy in diffusion-based image generation without retraining, by estimating counts and applying noise corrections during sampling.

Contribution

It introduces a model-agnostic, test-time framework with variants that improve count fidelity in diffusion models without altering their architectures.

Findings

Consistently improves object count accuracy, especially at higher counts.

Maintains favorable accuracy-runtime trade-offs across multiple diffusion models.

Effective on benchmarks and complex datasets.

Abstract

Text-to-image diffusion models achieve high visual fidelity but surprisingly exhibit systematic failures in numerical control when prompts specify explicit object counts. To address this limitation, we introduce ATHENA, a model-agnostic, test-time adaptive steering framework that improves object count fidelity without modifying model architectures or requiring retraining. ATHENA leverages intermediate representations during sampling to estimate object counts and applies count-aware noise corrections early in the denoising process, steering the generation trajectory before structural errors become difficult to revise. We present three progressively more advanced variants of ATHENA that trade additional computation for improved numerical accuracy, ranging from static prompt-based steering to dynamically adjusted count-aware control. Experiments on established benchmarks and a new visually and semantically complex dataset show that ATHENA consistently improves count fidelity, particularly at higher target counts, while maintaining favorable accuracy-runtime trade-offs across multiple diffusion backbones.