UniX: Unifying Autoregression and Diffusion for Chest X-Ray Understanding and Generation

TL;DR

UniX is a unified model that combines autoregressive understanding and diffusion-based generation for chest X-ray analysis, achieving high performance with fewer parameters by decoupling tasks and introducing cross-modal attention.

Contribution

The paper introduces UniX, a novel architecture that decouples understanding and generation tasks with specialized branches and cross-modal attention, enabling effective synergy and superior performance in medical imaging.

Findings

46.1% improvement in understanding (Micro-F1)

24.2% gain in generation quality (FD-RadDino)

Uses only a quarter of the parameters of LLM-CXR

Abstract

Despite recent progress, medical foundation models still struggle to unify visual understanding and generation, as these tasks have inherently conflicting goals: semantic abstraction versus pixel-level reconstruction. Existing approaches, typically based on parameter-shared autoregressive architectures, frequently lead to compromised performance in one or both tasks. To address this, we present UniX, a next-generation unified medical foundation model for chest X-ray understanding and generation. UniX decouples the two tasks into an autoregressive branch for understanding and a diffusion branch for high-fidelity generation. Crucially, a cross-modal self-attention mechanism is introduced to dynamically guide the generation process with understanding features. Coupled with a rigorous data cleaning pipeline and a multi-stage training strategy, this architecture enables synergistic collaboration between tasks while leveraging the strengths of diffusion models for superior generation. On two representative benchmarks, UniX achieves a 46.1% improvement in understanding performance (Micro-F1) and a 24.2% gain in generation quality (FD-RadDino), using only a quarter of the parameters of LLM-CXR. By achieving performance on par with task-specific models, our work establishes a scalable paradigm for synergistic medical image understanding and generation. Codes and models are available at https://github.com/ZrH42/UniX.