How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curation

TL;DR

This paper investigates the reliability issues in long-horizon multimodal image generation and introduces UniLongGen, a memory curation method that enhances stability and quality by actively forgetting interfering visual signals during inference.

Contribution

The paper identifies visual history pollution as a key problem and proposes UniLongGen, a novel, training-free inference strategy that dynamically curates memory to improve long-term generation fidelity.

Findings

UniLongGen improves long-horizon fidelity and consistency.

It reduces memory footprint and inference time.

Active forgetting is crucial for stable multimodal generation.

Abstract

Unified multimodal models hold the promise of generating extensive, interleaved narratives, weaving text and imagery into coherent long-form stories. However, current systems suffer from a critical reliability gap: as sequences grow, generation quality rapidly collapses. In this work, we investigate the mechanism behind this failure and argue that it is distinct from standard long-context challenges. We reveal that in generation, accumulated visual history acts as a source of active pollution, a decay governed specifically by the number of image events rather than raw token count. We identify a structural vulnerability where dense visual tokens overwhelm the attention mechanism, creating noise that distorts future synthesis. Guided by these mechanistic insights, we propose UniLongGen, a training-free inference strategy that prioritizes safe conditioning over total recall. Instead of retaining all history, UniLongGen dynamically curates the model's memory, identifying and discarding interfering visual signals based on the model's own internal relevance rankings. Extensive experiments demonstrate that this active forgetting approach is essential for stability: UniLongGen significantly outperforms baselines in long-horizon fidelity and consistency, while simultaneously reducing memory footprint and inference time.