Fluoroformer: Scaling multiple instance learning to multiplexed images via attention-based channel fusion

TL;DR

The paper introduces Fluoroformer, an attention-based MIL module designed for multiplexed whole slide images, demonstrating strong prognostic performance and biological interpretability in lung cancer analysis.

Contribution

It presents a novel MIL approach tailored for multiplexed WSIs using scaled dot-product attention for channel fusion, advancing AI application in spatial biology.

Findings

Achieves strong prognostic accuracy on NSCLC samples

Recapitulates key immuno-oncological features

Provides interpretability in multiplexed image analysis

Abstract

Though multiple instance learning (MIL) has been a foundational strategy in computational pathology for processing whole slide images (WSIs), current approaches are designed for traditional hematoxylin and eosin (H&E) slides rather than emerging multiplexed technologies. Here, we present an MIL strategy, the Fluoroformer module, that is specifically tailored to multiplexed WSIs by leveraging scaled dot-product attention (SDPA) to interpretably fuse information across disparate channels. On a cohort of 434 non-small cell lung cancer (NSCLC) samples, we show that the Fluoroformer both obtains strong prognostic performance and recapitulates immuno-oncological hallmarks of NSCLC. Our technique thereby provides a path for adapting state-of-the-art AI techniques to emerging spatial biology assays.