Meta-TTRL: A Metacognitive Framework for Self-Improving Test-Time Reinforcement Learning in Unified Multimodal Models

TL;DR

Meta-TTRL introduces a metacognitive test-time reinforcement learning framework that enables unified multimodal models to self-improve during inference, leading to significant performance gains across various tasks and models.

Contribution

It presents the first comprehensive framework for test-time reinforcement learning in unified multimodal models, leveraging model-intrinsic signals for self-optimization.

Findings

Meta-TTRL improves performance on compositional reasoning tasks.

It achieves significant gains on multiple T2I benchmarks.

The framework generalizes across different UMM architectures.

Abstract

Existing test-time scaling (TTS) methods for unified multimodal models (UMMs) in text-to-image (T2I) generation primarily rely on search or sampling strategies that produce only instance-level improvements, limiting the ability to learn from prior inferences and accumulate knowledge across similar prompts. To overcome these limitations, we propose Meta-TTRL, a metacognitive test-time reinforcement learning framework. Meta-TTRL performs test-time parameter optimization guided by model-intrinsic monitoring signals derived from the meta-knowledge of UMMs, achieving self-improvement and capability-level improvement at test time. Extensive experiments demonstrate that Meta-TTRL generalizes well across three representative UMMs, including Janus-Pro-7B, BAGEL, and Qwen-Image, achieving significant gains on compositional reasoning tasks and multiple T2I benchmarks with limited data. We provide the first comprehensive analysis to investigate the potential of test-time reinforcement learning (TTRL) for T2I generation in UMMs. Our analysis further reveals a key insight underlying effective TTRL: metacognitive synergy, where monitoring signals align with the model's optimization regime to enable self-improvement.