Improving RNA Secondary Structure Design using Deep Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach for RNA secondary structure design, demonstrating that DQN outperforms other algorithms in optimizing RNA sequences based on free energy minimization.

Contribution

It presents a novel benchmark applying reinforcement learning to RNA design and analyzes various algorithm variants and hyperparameters for improved performance.

Findings

DQN outperforms other RL algorithms in RNA design tasks.

Reward function modifications impact algorithm performance.

RL algorithms can effectively explore RNA sequence space.

Abstract

Rising costs in recent years of developing new drugs and treatments have led to extensive research in optimization techniques in biomolecular design. Currently, the most widely used approach in biomolecular design is directed evolution, which is a greedy hill-climbing algorithm that simulates biological evolution. In this paper, we propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. In addition to experimenting with the vanilla implementations of each reinforcement learning algorithm from standard libraries, we analyze variants of each algorithm in which we modify the algorithm's reward function and tune the model's hyperparameters. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches and its ability to search the possible space of RNA sequences. We find that our DQN algorithm performs by far the best in this setting, contrasting with, in which PPO performs the best among all tested algorithms. Our results should be of interest to those in the biomolecular design community and should serve as a baseline for future experiments involving machine learning in molecule design.