OWT: A Foundational Organ-Wise Tokenization Framework for Medical Imaging

TL;DR

The paper introduces OWT, a novel organ-wise tokenization framework for medical imaging that enhances interpretability, generalization, and clinical utility by explicitly disentangling images into organ-specific token groups.

Contribution

It presents a new tokenization approach with a training paradigm that explicitly separates organ information, improving interpretability and enabling advanced clinical applications.

Findings

OWT achieves strong performance on reconstruction and segmentation tasks.

It enables organ-specific tumor identification and semantic-level generation.

The framework enhances interpretability and clinical utility without extra training.

Abstract

Recent advances in representation learning often rely on holistic embeddings that entangle multiple semantic components, limiting interpretability and generalization. These issues are especially critical in medical imaging, where downstream tasks depend on anatomically interpretable features. To address these limitations, we propose an Organ-Wise Tokenization (OWT) framework with a Token Group-based Reconstruction (TGR) training paradigm. Unlike conventional approaches, OWT explicitly disentangles an image into separable token groups, each corresponding to a distinct organ or semantic entity. Our design ensures each token group encapsulates organ-specific information, boosting interpretability, generalization, and efficiency while enabling fine-grained control for targeted clinical applications. Experiments on CT and MRI datasets demonstrate OWT's power: it not only achieves strong performance on standard tasks like image reconstruction and segmentation, but also unlocks novel, high-impact clinical capabilities including organ-specific tumor identification, organ-level retrieval and semantic-level generation, without requiring any additional training. These findings underscore the potential of OWT as a foundational framework for semantically disentangled representation learning, offering broad scalability and a new perspective on how representations can be leveraged.