Gene-MOE: A sparsely gated prognosis and classification framework exploiting pan-cancer genomic information

TL;DR

Gene-MOE is a novel deep learning framework that leverages sparsely gated mixture of experts and attention mechanisms, pre-trained on pan-cancer data, to improve genomic analysis accuracy in cancer prognosis and classification.

Contribution

The paper introduces Gene-MOE, a new sparsely gated MOE framework with attention layers, pre-trained on pan-cancer data, enhancing analysis accuracy and addressing overfitting in genomic studies.

Findings

Gene-MOE outperformed state-of-the-art models in survival analysis on 12 of 14 cancer types.

Achieved 95.8% accuracy in classifying 33 cancer types, the best among compared models.

Learned rich, high-dimensional gene feature representations.

Abstract

Benefiting from the advancements in deep learning, various genomic analytical techniques, such as survival analysis, classification of tumors and their subtypes, and exploration of specific pathways, have significantly enhanced our understanding of the biological mechanisms driving cancer. However, the overfitting issue, arising from the limited number of patient samples, poses a challenge in improving the accuracy of genome analysis by deepening the neural network. Furthermore, it remains uncertain whether novel approaches such as the sparsely gated mixture of expert (MOE) and self-attention mechanisms can improve the accuracy of genomic analysis. In this paper, we introduce a novel sparsely gated RNA-seq analysis framework called Gene-MOE. This framework exploits the potential of the MOE layers and the proposed mixture of attention expert (MOAE) layers to enhance the analysis accuracy. Additionally, it addresses overfitting challenges by integrating pan-cancer information from 33 distinct cancer types through pre-training.We pre-trained Gene-MOE on TCGA pan-cancer RNA-seq dataset with 33 cancer types. Subsequently, we conducted experiments involving cancer classification and survival analysis based on the pre-trained Gene-MOE. According to the survival analysis results on 14 cancer types, Gene-MOE outperformed state-of-the-art models on 12 cancer types. Through detailed feature analysis, we found that the Gene-MOE model could learn rich feature representations of high-dimensional genes. According to the classification results, the total accuracy of the classification model for 33 cancer classifications reached 95.8%, representing the best performance compared to state-of-the-art models. These results indicate that Gene-MOE holds strong potential for use in cancer classification and survival analysis.