NLP Meets RNA: Unsupervised Embedding Learning for Ribozymes with Word2Vec

TL;DR

This paper applies Word2Vec, an NLP embedding technique, to learn meaningful vector representations of ribozymes, enabling improved classification and understanding of their structural features in bioinformatics.

Contribution

It introduces Ribo2Vec, the first application of Word2Vec to ribozyme sequences, demonstrating effective embeddings for classification and analysis.

Findings

Embeddings distinguish between ribozyme classes via PCA.

SVM classifier achieves promising accuracy with embeddings.

256D embeddings perform similarly to 128D, indicating lower dimensions suffice.

Abstract

Ribozymes, RNA molecules with distinct 3D structures and catalytic activity, have widespread applications in synthetic biology and therapeutics. However, relatively little research has focused on leveraging deep learning to enhance our understanding of ribozymes. This study implements Word2Vec, an unsupervised learning technique for natural language processing, to learn ribozyme embeddings. Ribo2Vec was trained on over 9,000 diverse ribozymes, learning to map sequences to 128 and 256-dimensional vector spaces. Using Ribo2Vec, sequence embeddings for five classes of ribozymes (hatchet, pistol, hairpin, hovlinc, and twister sister) were calculated. Principal component analysis demonstrated the ability of these embeddings to distinguish between ribozyme classes. Furthermore, a simple SVM classifier trained on ribozyme embeddings showed promising results in accurately classifying ribozyme types. Our results suggest that the embedding vectors contained meaningful information about ribozymes. Interestingly, 256-dimensional embeddings behaved similarly to 128-dimensional embeddings, suggesting that a lower dimension vector space is generally sufficient to capture ribozyme features. This approach demonstrates the potential of Word2Vec for bioinformatics, opening new avenues for ribozyme research. Future research includes using a Transformer-based method to learn RNA embeddings, which can capture long-range interactions between nucleotides.