Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modeling

TL;DR

Lumina-mGPT 2.0 is a fully autoregressive, stand-alone image generation model that achieves diffusion-model-level quality, supports multiple tasks, and offers flexible architecture and licensing, all trained from scratch.

Contribution

It introduces Lumina-mGPT 2.0, a novel autoregressive image model trained entirely from scratch, enabling versatile, high-quality, multi-task image generation without relying on pretrained components.

Findings

Matches state-of-the-art diffusion models in quality

Supports diverse tasks within a single framework

Outperforms diffusion models on standard benchmarks

Abstract

We present Lumina-mGPT 2.0, a stand-alone, decoder-only autoregressive model that revisits and revitalizes the autoregressive paradigm for high-quality image generation and beyond. Unlike existing approaches that rely on pretrained components or hybrid architectures, Lumina-mGPT 2.0 is trained entirely from scratch, enabling unrestricted architectural design and licensing freedom. It achieves generation quality on par with state-of-the-art diffusion models such as DALL-E 3 and SANA, while preserving the inherent flexibility and compositionality of autoregressive modeling. Our unified tokenization scheme allows the model to seamlessly handle a wide spectrum of tasks-including subject-driven generation, image editing, controllable synthesis, and dense prediction-within a single generative framework. To further boost usability, we incorporate efficient decoding strategies like inference-time scaling and speculative Jacobi sampling to improve quality and speed, respectively. Extensive evaluations on standard text-to-image benchmarks (e.g., GenEval, DPG) demonstrate that Lumina-mGPT 2.0 not only matches but in some cases surpasses diffusion-based models. Moreover, we confirm its multi-task capabilities on the Graph200K benchmark, with the native Lumina-mGPT 2.0 performing exceptionally well. These results position Lumina-mGPT 2.0 as a strong, flexible foundation model for unified multimodal generation. We have released our training details, code, and models at https://github.com/Alpha-VLLM/Lumina-mGPT-2.0.