Learning to Predict RNA Sequence Expressions from Whole Slide Images with Applications for Search and Classification

TL;DR

This paper introduces tRNAsfomer, an attention-based deep learning model that predicts RNA-seq data from whole slide images, enabling molecular insights without additional tissue sampling, and improves search and classification in digital pathology.

Contribution

The novel tRNAsfomer model combines multiple instance learning and attention mechanisms to predict molecular features directly from pathology images, outperforming existing methods.

Findings

Achieved better performance than state-of-the-art algorithms

Demonstrated faster convergence in experiments

Enabled molecular feature prediction from images

Abstract

Deep learning methods are widely applied in digital pathology to address clinical challenges such as prognosis and diagnosis. As one of the most recent applications, deep models have also been used to extract molecular features from whole slide images. Although molecular tests carry rich information, they are often expensive, time-consuming, and require additional tissue to sample. In this paper, we propose tRNAsfomer, an attention-based topology that can learn both to predict the bulk RNA-seq from an image and represent the whole slide image of a glass slide simultaneously. The tRNAsfomer uses multiple instance learning to solve a weakly supervised problem while the pixel-level annotation is not available for an image. We conducted several experiments and achieved better performance and faster convergence in comparison to the state-of-the-art algorithms. The proposed tRNAsfomer can assist as a computational pathology tool to facilitate a new generation of search and classification methods by combining the tissue morphology and the molecular fingerprint of the biopsy samples.