LoC-Path: Learning to Compress for Pathology Multimodal Large Language Models

TL;DR

LoC-Path introduces a resource-efficient multimodal large language model for pathology that compresses gigapixel slide features, reducing computational costs while maintaining competitive performance.

Contribution

The paper proposes LoC-Path, a novel architecture that compresses slide features using sparse token merging and importance scoring, enabling efficient end-to-end pathology modeling.

Findings

LoC-Path reduces inference latency and memory usage significantly.

It maintains competitive accuracy compared to existing slide-level MLLMs.

The approach enables practical deployment under limited computational resources.

Abstract

Whole Slide Image (WSI) MLLMs are difficult to build and deploy because gigapixel slides induce thousands of visual tokens, while only a small fraction of regions is diagnostically relevant. Existing slide-level pathology MLLMs typically combine heavy slide-level encoders with long visual prefixes, making end-to-end slide-level development and deployment expensive under limited computational resources. We revisit this regime and show that WSI tile features are highly redundant at both global and local scales, while task-relevant evidence is sparse and query-dependent. We therefore introduce LoC-Path, a resource-efficient slide-level MLLM that compresses before fusion. LoC-Path uses a Sparse Token Merger (STM) and an MAE-pretrained resampler to replace expensive slide-level encoding with a compact latent interface, then uses a Token Importance Scorer (TIS) to select the most relevant latents and a Cross-Attention Routing Adapter (CARA) to fuse them into a few LLM decoder layers. This design lowers both multimodal tuning cost and inference-time latency/memory by avoiding heavy slide-level encoding and long visual prefixes. Extensive experiments show that LoC-Path remains competitive with prior slide-level MLLMs while making end-to-end development and deployment more practical under limited computational resources.