# TVAE-RNA: ensemble-based RNA secondary structure prediction via transformer variational autoencoders

## TL;DR

TVAE-RNA is a new method for predicting RNA secondary structures using a combination of Transformer and VAE models, achieving better accuracy and diversity in predictions.

## Contribution

TVAE introduces a probabilistic framework combining Transformers and VAEs for RNA structure prediction, enabling diverse and biologically plausible outputs.

## Key findings

- TVAE achieves state-of-the-art performance with an F1 score of 0.89 and 83% accuracy.
- The model outperforms existing methods by 10% on benchmark datasets.
- GHA-Pairing enables fast and biologically constrained base-pairing for discrete predictions.

## Abstract

Accurate prediction of RNA secondary structure remains challenging due to the presence of pseudoknots, long-range dependencies, and limited labeled data.

We propose TVAE, a novel framework that integrates a Transformer encoder with a Variational Autoencoder (VAE). The Transformer captures global dependencies in the sequence, while the VAE models structural variability by learning a probabilistic latent space. Unlike deterministic models, TVAE generates diverse and biologically plausible secondary structures, enabling more comprehensive structure discovery. To obtain discrete predictions, we introduce GHA-Pairing, a fast and biologically constrained base-pairing algorithm. TVAE demonstrates strong generalization across different RNA families and achieves state-of-the-art performance on benchmark datasets, reaching an F1 score of 0.89 and 83% accuracy, surpassing existing methods by 10%. These results highlight the advantage of probabilistic modeling for RNA structure prediction and its potential to enhance biological insights.

Code and pretrained models are available at https://github.com/mei-rna/TVAE-RNA. The released version of the dataset and models can also be accessed via DOI: 10.5281/zenodo.16946114.

Graphical Abstract