Predicting Distance matrix with large language models

TL;DR

This paper demonstrates that large pretrained RNA language models combined with transformers can accurately predict RNA base distance maps using only primary sequence data, aiding structural understanding.

Contribution

The study introduces a novel approach leveraging pretrained RNA language models and transformers to predict RNA distance maps solely from sequence data.

Findings

Accurate RNA base distance predictions achieved from sequence alone.

Large pretrained models effectively capture structural information.

Method improves RNA structural modeling accuracy.

Abstract

Structural prediction has long been considered critical in RNA research, especially following the success of AlphaFold2 in protein studies, which has drawn significant attention to the field. While recent advances in machine learning and data accumulation have effectively addressed many biological tasks, particularly in protein related research. RNA structure prediction remains a significant challenge due to data limitations. Obtaining RNA structural data is difficult because traditional methods such as nuclear magnetic resonance spectroscopy, Xray crystallography, and electron microscopy are expensive and time consuming. Although several RNA 3D structure prediction methods have been proposed, their accuracy is still limited. Predicting RNA structural information at another level, such as distance maps, remains highly valuable. Distance maps provide a simplified representation of spatial constraints between nucleotides, capturing essential relationships without requiring a full 3D model. This intermediate level of structural information can guide more accurate 3D modeling and is computationally less intensive, making it a useful tool for improving structural predictions. In this work, we demonstrate that using only primary sequence information, we can accurately infer the distances between RNA bases by utilizing a large pretrained RNA language model coupled with a well trained downstream transformer.