Context Unrolling in Omni Models

TL;DR

Omni is a unified multimodal model trained on diverse data types, enabling explicit reasoning across modalities through Context Unrolling, which enhances multimodal understanding and reasoning capabilities.

Contribution

The paper introduces Context Unrolling in Omni, a novel approach for explicit multimodal reasoning within a unified model architecture.

Findings

Omni achieves strong performance on multimodal benchmarks.

Omni demonstrates advanced reasoning across text, images, videos, and 3D geometry.

Context Unrolling improves the model's ability to aggregate heterogeneous information.

Abstract

We present Omni, a unified multimodal model natively trained on diverse modalities, including text, images, videos, 3D geometry, and hidden representations. We find that such training enables Context Unrolling, where the model explicitly reasons across multiple modal representations before producing predictions. This process enables the model to aggregate complementary information across heterogeneous modalities, facilitating a more faithful approximation of the shared multimodal knowledge manifold and improving downstream reasoning fidelity. As a result, Omni achieves strong performance on both multimodal generation and understanding benchmarks, while demonstrating advanced multimodal reasoning capabilities, including in-context generation of text, image, video, and 3D geometry.