OmniGen2: Towards Instruction-Aligned Multimodal Generation

TL;DR

OmniGen2 is an open-source multimodal generative model that unifies text-to-image, editing, and in-context tasks with innovative decoding pathways and reflection mechanisms, achieving state-of-the-art open-source performance.

Contribution

It introduces a novel multimodal model with separate decoding pathways and a reflection mechanism, enhancing versatility and performance across diverse generation tasks.

Findings

Achieves competitive results on multiple benchmarks.

Sets state-of-the-art among open-source models for consistency.

Supports diverse tasks with a unified architecture.

Abstract

In this work, we introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generation tasks, including text-to-image, image editing, and in-context generation. Unlike OmniGen v1, OmniGen2 features two distinct decoding pathways for text and image modalities, utilizing unshared parameters and a decoupled image tokenizer. This design enables OmniGen2 to build upon existing multimodal understanding models without the need to re-adapt VAE inputs, thereby preserving the original text generation capabilities. To facilitate the training of OmniGen2, we developed comprehensive data construction pipelines, encompassing image editing and in-context generation data. Additionally, we introduce a reflection mechanism tailored for image generation tasks and curate a dedicated reflection dataset based on OmniGen2. Despite its relatively modest parameter size, OmniGen2 achieves competitive results on multiple task benchmarks, including text-to-image and image editing. To further evaluate in-context generation, also referred to as subject-driven tasks, we introduce a new benchmark named OmniContext. OmniGen2 achieves state-of-the-art performance among open-source models in terms of consistency. We will release our models, training code, datasets, and data construction pipeline to support future research in this field. Project Page: https://vectorspacelab.github.io/OmniGen2; GitHub Link: https://github.com/VectorSpaceLab/OmniGen2