DreamVAR: Taming Reinforced Visual Autoregressive Model for High-Fidelity Subject-Driven Image Generation

TL;DR

DreamVAR introduces a novel visual autoregressive model for subject-driven image generation, leveraging multi-scale features and reinforcement learning to improve image quality and subject consistency over existing diffusion methods.

Contribution

The paper presents DreamVAR, a new VAR-based framework that simplifies autoregressive dependencies and enhances subject fidelity using reinforcement learning.

Findings

Outperforms diffusion models in appearance preservation.

Simplifies autoregressive dependencies with pre-filled subject features.

Uses reinforcement learning to improve semantic alignment.

Abstract

Recent advances in subject-driven image generation using diffusion models have attracted considerable attention for their remarkable capabilities in producing high-quality images. Nevertheless, the potential of Visual Autoregressive (VAR) models, despite their unified architecture and efficient inference, remains underexplored. In this work, we present DreamVAR, a novel framework for subject-driven image synthesis built upon a VAR model that employs next-scale prediction. Technically, multi-scale features of the reference subject are first extracted by a visual tokenizer. Instead of interleaving these conditional features with target image tokens across scales, our DreamVAR pre-fills the full subject feature sequence prior to predicting target image tokens. This design simplifies autoregressive dependencies and mitigates the train-test discrepancy in multi-scale conditioning scenario within the VAR paradigm. DreamVAR further incorporates reinforcement learning to jointly enhance semantic alignment and subject consistency. Extensive experiments demonstrate that DreamVAR achieves superior appearance preservation compared to leading diffusion-based methods.