Histopath-C: Towards Realistic Domain Shifts for Histopathology Vision-Language Adaptation

TL;DR

This paper introduces Histopath-C, a benchmark with realistic corruptions for histopathology images, and proposes LATTE, a novel adaptation method that improves vision-language model robustness against domain shifts in medical imaging.

Contribution

The paper presents a new benchmark for realistic histopathology corruptions and a low-rank adaptation strategy called LATTE that enhances model robustness to domain shifts.

Findings

LATTE outperforms existing TTA methods on histopathology datasets.

The benchmark effectively simulates real-world distribution shifts.

LATTE reduces sensitivity to diverse text inputs in VLMs.

Abstract

Medical Vision-language models (VLMs) have shown remarkable performances in various medical imaging domains such as histo\-pathology by leveraging pre-trained, contrastive models that exploit visual and textual information. However, histopathology images may exhibit severe domain shifts, such as staining, contamination, blurring, and noise, which may severely degrade the VLM's downstream performance. In this work, we introduce Histopath-C, a new benchmark with realistic synthetic corruptions designed to mimic real-world distribution shifts observed in digital histopathology. Our framework dynamically applies corruptions to any available dataset and evaluates Test-Time Adaptation (TTA) mechanisms on the fly. We then propose LATTE, a transductive, low-rank adaptation strategy that exploits multiple text templates, mitigating the sensitivity of histopathology VLMs to diverse text inputs. Our approach outperforms state-of-the-art TTA methods originally designed for natural images across a breadth of histopathology datasets, demonstrating the effectiveness of our proposed design for robust adaptation in histopathology images. Code and data are available at https://github.com/Mehrdad-Noori/Histopath-C.