UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation

TL;DR

UniGen is a unified multimodal large language model that enhances image understanding and generation through innovative training strategies and a novel test-time verification method, achieving state-of-the-art results.

Contribution

The paper introduces UniGen, a fully open-source trained MLLM with a new Chain-of-Thought Verification strategy for improved image generation quality.

Findings

Achieves state-of-the-art scores on GenEval and DPG-Bench benchmarks.

Proposes a test-time verification method that improves semantic alignment.

Provides insights into the full lifecycle of building unified MLLMs.

Abstract

We introduce UniGen, a unified multimodal large language model (MLLM) capable of image understanding and generation. We study the full training pipeline of UniGen from a data-centric perspective, including multi-stage pre-training, supervised fine-tuning, and direct preference optimization. More importantly, we propose a new Chain-of-Thought Verification (CoT-V) strategy for test-time scaling, which significantly boosts UniGen's image generation quality using a simple Best-of-N test-time strategy. Specifically, CoT-V enables UniGen to act as both image generator and verifier at test time, assessing the semantic alignment between a text prompt and its generated image in a step-by-step CoT manner. Trained entirely on open-source datasets across all stages, UniGen achieves state-of-the-art performance on a range of image understanding and generation benchmarks, with a final score of 0.78 on GenEval and 85.19 on DPG-Bench. Through extensive ablation studies, our work provides actionable insights and addresses key challenges in the full life cycle of building unified MLLMs, contributing meaningful directions to the future research.