ThinkGen: Generalized Thinking for Visual Generation

TL;DR

ThinkGen introduces a novel framework that combines Chain-of-Thought reasoning with multimodal large language models and diffusion transformers to enhance generalization and quality in visual generation tasks.

Contribution

It is the first to explicitly leverage CoT reasoning in visual generation, employing a decoupled architecture and a novel training paradigm for diverse scenarios.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Demonstrates effective reasoning in diverse generative tasks.

Flexible joint training across datasets enhances performance.

Abstract

Recent progress in Multimodal Large Language Models (MLLMs) demonstrates that Chain-of-Thought (CoT) reasoning enables systematic solutions to complex understanding tasks. However, its extension to generation tasks remains nascent and limited by scenario-specific mechanisms that hinder generalization and adaptation. In this work, we present ThinkGen, the first think-driven visual generation framework that explicitly leverages MLLM's CoT reasoning in various generation scenarios. ThinkGen employs a decoupled architecture comprising a pretrained MLLM and a Diffusion Transformer (DiT), wherein the MLLM generates tailored instructions based on user intent, and DiT produces high-quality images guided by these instructions. We further propose a separable GRPO-based training paradigm (SepGRPO), alternating reinforcement learning between the MLLM and DiT modules. This flexible design enables joint training across diverse datasets, facilitating effective CoT reasoning for a wide range of generative scenarios. Extensive experiments demonstrate that ThinkGen achieves robust, state-of-the-art performance across multiple generation benchmarks. Code is available: https://github.com/jiaosiyuu/ThinkGen