Modular Multi-Task Learning for Chemical Reaction Prediction

TL;DR

This paper evaluates Low-Rank Adaptation (LoRA) as a parameter-efficient method for adapting large language models to specific chemical reaction prediction tasks, demonstrating comparable accuracy to full fine-tuning and better preservation of multi-task performance.

Contribution

It introduces LoRA as an effective, modular fine-tuning approach for chemical reaction prediction, reducing computational costs and mitigating catastrophic forgetting in large language models.

Findings

LoRA achieves accuracy comparable to full fine-tuning.

LoRA better preserves multi-task performance.

Both methods generalize beyond training data.

Abstract

Adapting large language models (LLMs) trained on broad organic chemistry to smaller, domain-specific reaction datasets is a key challenge in chemical and pharmaceutical R&D. Effective specialisation requires learning new reaction knowledge while preserving general chemical understanding across related tasks. Here, we evaluate Low-Rank Adaptation (LoRA) as a parameter-efficient alternative to full fine-tuning for organic reaction prediction on limited, complex datasets. Using USPTO reaction classes and challenging C-H functionalisation reactions, we benchmark forward reaction prediction, retrosynthesis and reagent prediction. LoRA achieves accuracy comparable to full fine-tuning while effectively mitigating catastrophic forgetting and better preserving multi-task performance. Both fine-tuning approaches generalise beyond training distributions, producing plausible alternative solvent predictions. Notably, C-H functionalisation fine-tuning reveals that LoRA and full fine-tuning encode subtly different reactivity patterns, suggesting more effective reaction-specific adaptation with LoRA. As LLMs continue to scale, our results highlight the practicality of modular, parameter-efficient fine-tuning strategies for their flexible deployment for chemistry applications.