SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive Generation

TL;DR

This paper introduces Scaled Spatial Guidance (SSG), a training-free inference technique that enhances multi-scale visual autoregressive models by emphasizing high-frequency details, improving image fidelity and diversity without increasing latency.

Contribution

The paper proposes SSG, a novel inference-time guidance method that maintains the coarse-to-fine hierarchy in VAR models by isolating and emphasizing high-frequency signals, improving image quality.

Findings

SSG improves image fidelity and diversity in VAR models.

SSG maintains low latency during inference.

The method is broadly applicable across different tokenization and conditioning modalities.

Abstract

Visual autoregressive (VAR) models generate images through next-scale prediction, naturally achieving coarse-to-fine, fast, high-fidelity synthesis mirroring human perception. In practice, this hierarchy can drift at inference time, as limited capacity and accumulated error cause the model to deviate from its coarse-to-fine nature. We revisit this limitation from an information-theoretic perspective and deduce that ensuring each scale contributes high-frequency content not explained by earlier scales mitigates the train-inference discrepancy. With this insight, we propose Scaled Spatial Guidance (SSG), training-free, inference-time guidance that steers generation toward the intended hierarchy while maintaining global coherence. SSG emphasizes target high-frequency signals, defined as the semantic residual, isolated from a coarser prior. To obtain this prior, we leverage a principled frequency-domain procedure, Discrete Spatial Enhancement (DSE), which is devised to sharpen and better isolate the semantic residual through frequency-aware construction. SSG applies broadly across VAR models leveraging discrete visual tokens, regardless of tokenization design or conditioning modality. Experiments demonstrate SSG yields consistent gains in fidelity and diversity while preserving low latency, revealing untapped efficiency in coarse-to-fine image generation. Code is available at https://github.com/Youngwoo-git/SSG.