Prompting Whole Slide Image Based Genetic Biomarker Prediction

TL;DR

This paper introduces a novel method for predicting genetic biomarkers from whole slide images of colorectal cancer using prompting techniques and large language models, improving accuracy and interpretability.

Contribution

It presents a new approach leveraging large language models for generating prompts to extract and analyze pathological components related to genetic biomarkers in WSIs.

Findings

Achieved 91.49% AUC in MSI classification

Demonstrated clinical interpretability of the method

Outperformed existing approaches on two datasets

Abstract

Prediction of genetic biomarkers, e.g., microsatellite instability and BRAF in colorectal cancer is crucial for clinical decision making. In this paper, we propose a whole slide image (WSI) based genetic biomarker prediction method via prompting techniques. Our work aims at addressing the following challenges: (1) extracting foreground instances related to genetic biomarkers from gigapixel WSIs, and (2) the interaction among the fine-grained pathological components in WSIs.Specifically, we leverage large language models to generate medical prompts that serve as prior knowledge in extracting instances associated with genetic biomarkers. We adopt a coarse-to-fine approach to mine biomarker information within the tumor microenvironment. This involves extracting instances related to genetic biomarkers using coarse medical prior knowledge, grouping pathology instances into fine-grained pathological components and mining their interactions. Experimental results on two colorectal cancer datasets show the superiority of our method, achieving 91.49% in AUC for MSI classification. The analysis further shows the clinical interpretability of our method. Code is publicly available at https://github.com/DeepMed-Lab-ECNU/PromptBio.