Cross-Organ and Cross-Scanner Adenocarcinoma Segmentation using Rein to Fine-tune Vision Foundation Models

TL;DR

This paper introduces Rein, a novel fine-tuning method using learnable tokens, to adapt vision foundation models for cross-organ and cross-scanner adenocarcinoma segmentation, addressing domain discrepancies in digital pathology images.

Contribution

The paper proposes Rein, a new fine-tuning approach with learnable tokens, to improve vision foundation models for digital pathology segmentation across different organs and scanners.

Findings

Rein effectively fine-tunes VFMs for adenocarcinoma segmentation.

Achieved high scores on MICCAI 2024 COSAS challenge tasks.

Demonstrated robustness across diverse domain variations.

Abstract

In recent years, significant progress has been made in tumor segmentation within the field of digital pathology. However, variations in organs, tissue preparation methods, and image acquisition processes can lead to domain discrepancies among digital pathology images. To address this problem, in this paper, we use Rein, a fine-tuning method, to parametrically and efficiently fine-tune various vision foundation models (VFMs) for MICCAI 2024 Cross-Organ and Cross-Scanner Adenocarcinoma Segmentation (COSAS2024). The core of Rein consists of a set of learnable tokens, which are directly linked to instances, improving functionality at the instance level in each layer. In the data environment of the COSAS2024 Challenge, extensive experiments demonstrate that Rein fine-tuned the VFMs to achieve satisfactory results. Specifically, we used Rein to fine-tune ConvNeXt and DINOv2. Our team used the former to achieve scores of 0.7719 and 0.7557 on the preliminary test phase and final test phase in task1, respectively, while the latter achieved scores of 0.8848 and 0.8192 on the preliminary test phase and final test phase in task2. Code is available at GitHub.