Explainable AI in Genomics: Transcription Factor Binding Site Prediction with Mixture of Experts

TL;DR

This paper presents a novel Mixture of Experts model for transcription factor binding site prediction that outperforms individual models, especially in out-of-distribution scenarios, and introduces a new interpretability technique called ShiftSmooth.

Contribution

The study introduces a Mixture of Experts approach combining multiple CNN models and a new attribution method, ShiftSmooth, for improved accuracy and interpretability in TFBS prediction.

Findings

MoE model outperforms individual experts on diverse datasets

ShiftSmooth provides more robust attribution than Vanilla Gradient

ANOVA confirms significant performance improvements

Abstract

Transcription Factor Binding Site (TFBS) prediction is crucial for understanding gene regulation and various biological processes. This study introduces a novel Mixture of Experts (MoE) approach for TFBS prediction, integrating multiple pre-trained Convolutional Neural Network (CNN) models, each specializing in different TFBS patterns. We evaluate the performance of our MoE model against individual expert models on both in-distribution and out-of-distribution (OOD) datasets, using six randomly selected transcription factors (TFs) for OOD testing. Our results demonstrate that the MoE model achieves competitive or superior performance across diverse TF binding sites, particularly excelling in OOD scenarios. The Analysis of Variance (ANOVA) statistical test confirms the significance of these performance differences. Additionally, we introduce ShiftSmooth, a novel attribution mapping technique that provides more robust model interpretability by considering small shifts in input sequences. Through comprehensive explainability analysis, we show that ShiftSmooth offers superior attribution for motif discovery and localization compared to traditional Vanilla Gradient methods. Our work presents an efficient, generalizable, and interpretable solution for TFBS prediction, potentially enabling new discoveries in genome biology and advancing our understanding of transcriptional regulation.