Genomic Interpreter: A Hierarchical Genomic Deep Neural Network with 1D Shifted Window Transformer

TL;DR

Genomic Interpreter is a hierarchical deep neural network with a novel 1D Shifted Window Transformer that improves genomic assay prediction and reveals hierarchical dependencies in genomic data.

Contribution

It introduces a new Transformer-based architecture, 1D-Swin, for modeling long-range hierarchical genomic data, outperforming existing models.

Findings

Outperforms state-of-the-art models in genomic assay prediction

Identifies hierarchical dependencies in genomic sites

Unveils the underlying 'syntax' of gene regulation

Abstract

Given the increasing volume and quality of genomics data, extracting new insights requires interpretable machine-learning models. This work presents Genomic Interpreter: a novel architecture for genomic assay prediction. This model outperforms the state-of-the-art models for genomic assay prediction tasks. Our model can identify hierarchical dependencies in genomic sites. This is achieved through the integration of 1D-Swin, a novel Transformer-based block designed by us for modelling long-range hierarchical data. Evaluated on a dataset containing 38,171 DNA segments of 17K base pairs, Genomic Interpreter demonstrates superior performance in chromatin accessibility and gene expression prediction and unmasks the underlying `syntax' of gene regulation.