# DRFormer: A Benchmark Model for RNA Sequence Downstream Tasks

## TL;DR

DRFormer is a new model for RNA analysis that uses structural features and multimodal learning to improve prediction accuracy across various RNA tasks.

## Contribution

DRFormer is the first RNA model to integrate structural vision features with sequence data in a multimodal framework.

## Key findings

- Achieved 94.4% MCC in RNA sequence classification, surpassing RNAErnie by 1.2%.
- Outperformed BERT-RBP and PrismNet in protein–RNA interaction prediction with an MCC of 0.492.
- Exceeded SPOT-RNA by 1% in RNA secondary structure prediction with an F1 score of 0.690.

## Abstract

Background/Objectives: RNA research is critical for understanding gene regulation, disease mechanisms, and therapeutic development. Constructing effective RNA benchmark models for accurate downstream analysis has become a significant research challenge. The objective of this study is to propose a robust benchmark model, DRFormer, for RNA sequence downstream tasks. Methods: The DRFormer model utilizes RNA sequences to construct novel vision features based on secondary structure and sequence distance. These features are pre-trained using the SWIN model to develop a SWIN-RNA submodel. This submodel is then integrated with an RNA sequence model to construct a multimodal model for downstream analysis. Results: We conducted experiments on various RNA downstream tasks. In the sequence classification task, the MCC reached 94.4%, surpassing the state-of-the-art RNAErnie model by 1.2%. In the protein–RNA interaction prediction, DRFormer achieved an MCC of 0.492, outperforming advanced models like BERT-RBP and PrismNet. In RNA secondary structure prediction, the F1 score was 0.690, exceeding the widely used SPOT-RNA model by 1%. Additionally, generalization experiments on DNA tasks yielded satisfactory results. Conclusions: DRFormer is the first RNA sequence downstream analysis model that leverages structural features to construct a vision model and integrates sequence and vision models in a multimodal manner. This approach yields excellent prediction and analysis results, making it a valuable contribution to RNA research.