C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs

TL;DR

C-MORAL is a reinforcement learning framework that improves controllable multi-objective molecular optimization with LLMs, outperforming state-of-the-art models on benchmark tasks.

Contribution

It introduces a novel RL post-training method combining group-based optimization and property score alignment for better multi-objective control.

Findings

Achieves the highest Success Optimized Rate (SOR) of 48.9% on IND tasks.

Outperforms existing models on both in-domain and out-of-domain benchmarks.

Largely preserves scaffold similarity during optimization.

Abstract

Large language models (LLMs) show promise for molecular optimization, but aligning them with selective and competing drug-design constraints remains challenging. We propose C-Moral, a reinforcement learning post-training framework for controllable multi-objective molecular optimization. C-Moral combines group-based relative optimization, property score alignment for heterogeneous objectives, and continuous non-linear reward aggregation to improve stability across competing properties. Experiments on the C-MuMOInstruct benchmark show that C-Moral consistently outperforms state-of-the-art models across both in-domain and out-of-domain settings, achieving the best Success Optimized Rate (SOR) of 48.9% on IND tasks and 39.5% on OOD tasks, while largely preserving scaffold similarity. These results suggest that RL post-training is an effective way to align molecular language models with continuous molecular design objectives. Our code and models are publicly available at https://github.com/Rwigie/C-MORAL.