LoRA-BERT: a Natural Language Processing Model for Robust and Accurate Prediction of long non-coding RNAs

TL;DR

LoRA-BERT is a novel transformer-based model that improves the robustness and accuracy of long non-coding RNA classification, outperforming existing tools in biological sequence prediction tasks.

Contribution

Introduction of LoRA-BERT, a pre-trained bidirectional encoder that captures nucleotide-level information for enhanced lncRNA and mRNA prediction accuracy.

Findings

LoRA-BERT outperforms existing sequence prediction tools in accuracy and efficiency.

Achieves state-of-the-art performance in predicting human and mouse lncRNAs and mRNAs.

Provides insights into the traits of lncRNAs and mRNAs relevant to disease detection.

Abstract

Long non-coding RNAs (lncRNAs) serve as crucial regulators in numerous biological processes. Although they share sequence similarities with messenger RNAs (mRNAs), lncRNAs perform entirely different roles, providing new avenues for biological research. The emergence of next-generation sequencing technologies has greatly advanced the detection and identification of lncRNA transcripts and deep learning-based approaches have been introduced to classify long non-coding RNAs (lncRNAs). These advanced methods have significantly enhanced the efficiency of identifying lncRNAs. However, many of these methods are devoid of robustness and accuracy due to the extended length of the sequences involved. To tackle this issue, we have introduced a novel pre-trained bidirectional encoder representation called LoRA-BERT. LoRA-BERT is designed to capture the importance of nucleotide-level information during sequence classification, leading to more robust and satisfactory outcomes. In a comprehensive comparison with commonly used sequence prediction tools, we have demonstrated that LoRA-BERT outperforms them in terms of accuracy and efficiency. Our results indicate that, when utilizing the transformer model, LoRA-BERT achieves state-of-the-art performance in predicting both lncRNAs and mRNAs for human and mouse species. Through the utilization of LoRA-BERT, we acquire valuable insights into the traits of lncRNAs and mRNAs, offering the potential to aid in the comprehension and detection of diseases linked to lncRNAs in humans.