Histopathology image embedding based on foundation models features aggregation for patient treatment response prediction

TL;DR

This paper introduces a novel approach using foundation models and attention-based aggregation of histopathology image features to predict treatment response in lymphoma patients, demonstrating improved performance over traditional methods.

Contribution

The study presents a new methodology leveraging foundation models for feature extraction and attention-based multiple instance learning for treatment response prediction from histopathology images.

Findings

Foundation models outperform ImageNet pre-training in this task.

The method achieves promising results on a dataset of 152 patients.

Foundation models effectively characterize histopathology images for predictive tasks.

Abstract

Predicting the response of a patient to a cancer treatment is of high interest. Nonetheless, this task is still challenging from a medical point of view due to the complexity of the interaction between the patient organism and the considered treatment. Recent works on foundation models pre-trained with self-supervised learning on large-scale unlabeled histopathology datasets have opened a new direction towards the development of new methods for cancer diagnosis related tasks. In this article, we propose a novel methodology for predicting Diffuse Large B-Cell Lymphoma patients treatment response from Whole Slide Images. Our method exploits several foundation models as feature extractors to obtain a local representation of the image corresponding to a small region of the tissue, then, a global representation of the image is obtained by aggregating these local representations using attention-based Multiple Instance Learning. Our experimental study conducted on a dataset of 152 patients, shows the promising results of our methodology, notably by highlighting the advantage of using foundation models compared to conventional ImageNet pre-training. Moreover, the obtained results clearly demonstrates the potential of foundation models for characterizing histopathology images and generating more suited semantic representation for this task.