RiboPO: Preference Optimization for Structure- and Stability-Aware RNA Design

TL;DR

RiboPO introduces a reinforcement learning framework that optimizes RNA sequences for accurate 3D structure, stability, and global geometry, outperforming existing methods on key benchmarks.

Contribution

It presents RiboPO, a novel preference optimization approach using reinforcement learning from physical feedback for multi-objective RNA design.

Findings

Improves MFE by 12.3% over baselines

Increases secondary-structure self-consistency by 20%

Achieves 11% higher pass@64 sampling efficiency

Abstract

Designing RNA sequences that reliably adopt specified three-dimensional structures while maintaining thermodynamic stability remains challenging for synthetic biology and therapeutics. Current inverse folding approaches optimize for sequence recovery or single structural metrics, failing to simultaneously ensure global geometry, local accuracy, and ensemble stability-three interdependent requirements for functional RNA design. This gap becomes critical when designed sequences encounter dynamic biological environments. We introduce RiboPO, a Ribonucleic acid Preference Optimization framework that addresses this multi-objective challenge through reinforcement learning from physical feedback (RLPF). RiboPO fine-tunes gRNAde by constructing preference pairs from composite physical criteria that couple global 3D fidelity and thermodynamic stability. Preferences are formed using structural gates, PLDDT geometry assessments, and thermostability proxies with variability-aware margins, and the policy is updated with Direct Preference Optimization (DPO). On RNA inverse folding benchmarks, RiboPO demonstrates a superior balance of structural accuracy and stability. Compared to the best non-overlap baselines, our multi-round model improves Minimum Free Energy (MFE) by 12.3% and increases secondary-structure self-consistency (EternaFold scMCC) by 20%, while maintaining competitive 3D quality and high sequence diversity. In sampling efficiency, RiboPO achieves 11% higher pass@64 than the gRNAde base under the conjunction of multiple requirements. A multi-round variant with preference-pair reconstruction delivers additional gains on unseen RNA structures. These results establish RLPF as an effective paradigm for structure-accurate and ensemble-robust RNA design, providing a foundation for extending to complex biological objectives.