UAM: A Unified Attention-Mamba Backbone of Multimodal Framework for Tumor Cell Classification

TL;DR

This paper introduces a novel unified backbone architecture called UAM that combines attention and Mamba modules for improved multimodal tumor cell classification and segmentation, achieving state-of-the-art results.

Contribution

The paper presents a unified Attention-Mamba backbone that adaptively integrates modules, eliminating manual tuning and enhancing performance in multimodal tumor cell analysis.

Findings

UAM achieves 78% cell classification accuracy, up from 74%.

UAM improves tumor segmentation precision to 80%.

State-of-the-art performance on public benchmarks.

Abstract

Inspired by the recent success of the Mamba architecture in vision and language domains, we introduce a Unified Attention-Mamba (UAM) backbone. Unlike previous hybrid approaches that integrate Attention and Mamba modules in fixed proportions, our unified design flexibly combines their capabilities within a single cohesive architecture, eliminating the need for manual ratio tuning and improving encode capability. We develop two UAM variants to comprehensively evaluate the benefits of this unified structure. Building on this backbone, we further propose a multimodal UAM framework that jointly performs cell-level classification and image segmentation. Experimental results demonstrate that UAM achieves state-of-the-art performance across both tasks on public benchmarks, surpassing leading image-based foundation models. It improves cell classification accuracy from 74\% to 78\% ($n$=349,882 cells), and tumor segmentation precision from 75\% to 80\% ($n$=406 patches).