Compact Artificial Neural Network Models for Predicting Protein Residue -- RNA Base Binding

TL;DR

This study demonstrates that small, shallow neural networks with appropriate training techniques can effectively predict protein-RNA binding, making such predictions accessible without extensive computational resources.

Contribution

The paper introduces a simple, resource-efficient neural network approach for protein-RNA binding prediction, challenging the need for large models and extensive data.

Findings

ReLU, GLU, and Tanh are effective non-linearities for small ANNs.

Increasing training data and ensemble methods improve accuracy.

Optimal context window size is approximately 30 residues and bases.

Abstract

Large Artificial Neural Network (ANN) models have demonstrated success in various domains, including general text and image generation, drug discovery, and protein-RNA (ribonucleic acid) binding tasks. However, these models typically demand substantial computational resources, time, and data for effective training. Given that such extensive resources are often inaccessible to many researchers and that life sciences data sets are frequently limited, we investigated whether small ANN models could achieve acceptable accuracy in protein-RNA prediction. We experimented with shallow feed-forward ANNs comprising two hidden layers and various non-linearities. These models did not utilize explicit structural information; instead, a sliding window approach was employed to implicitly consider the context of neighboring residues and bases. We explored different training techniques to address the issue of highly unbalanced data. Among the seven most popular non-linearities for feed-forward ANNs, only three: Rectified Linear Unit (ReLU), Gated Linear Unit (GLU), and Hyperbolic Tangent (Tanh) yielded converging models. Common re-balancing techniques, such as under- and over-sampling of training sets, proved ineffective, whereas increasing the volume of training data and using model ensembles significantly improved performance. The optimal context window size, balancing both false negative and false positive errors, was found to be approximately 30 residues and bases. Our findings indicate that high-accuracy protein-RNA binding prediction is achievable using computing hardware accessible to most educational and research institutions.