BioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data

TL;DR

BioAlchemy introduces a large, curated dataset of biology reasoning problems and demonstrates how aligning training data with current research topics enhances reinforcement learning models' reasoning performance in biology.

Contribution

The paper presents BioAlchemy, a pipeline for extracting biology reasoning questions, creating a large dataset, and improving model performance by aligning data with modern research topics.

Findings

BioAlchemy-345K contains over 345,000 biology reasoning problems.

Aligning dataset topics with current biology research improves model reasoning performance.

BioAlchemist-8B outperforms its base model by 9.12% on biology benchmarks.

Abstract

Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology questions from current large-scale reasoning datasets do not align well with modern research topic distributions in biology, and that this topic imbalance may negatively affect performance. In addition, we find that methods for extracting challenging and verifiable research problems from biology research text are a critical yet underdeveloped ingredient in applying reinforcement learning for better performance on biology research tasks. We introduce BioAlchemy, a pipeline for sourcing a diverse set of verifiable question-and-answer pairs from a scientific corpus of biology research text. We curate BioAlchemy-345K, a training dataset containing over 345K scientific reasoning problems in biology. Then, we demonstrate how aligning our dataset to the topic distribution of modern scientific biology can be used with reinforcement learning to improve reasoning performance. Finally, we present BioAlchemist-8B, which improves over its base reasoning model by 9.12% on biology benchmarks. These results demonstrate the efficacy of our approach for developing stronger scientific reasoning capabilities in biology. The BioAlchemist-8B model is available at: https://huggingface.co/BioAlchemy.