BioDisco: Multi-agent hypothesis generation with dual-mode evidence, iterative feedback and temporal evaluation

TL;DR

BioDisco is a multi-agent framework that leverages language models, dual evidence sources, and iterative feedback to generate and refine novel, evidence-grounded hypotheses in biomedical research, with rigorous temporal and human evaluation.

Contribution

We introduce BioDisco, a novel multi-agent system combining dual evidence modes, iterative refinement, and temporal evaluation for improved hypothesis generation in biomedicine.

Findings

BioDisco outperforms ablated models and generalist agents in novelty and significance.

It demonstrates effective iterative refinement and temporal evaluation.

The framework is flexible, modular, and easy to integrate with custom models.

Abstract

Identifying novel hypotheses is essential to scientific research, yet this process risks being overwhelmed by the sheer volume and complexity of available information. Existing automated methods often struggle to generate novel and evidence-grounded hypotheses, lack robust iterative refinement and rarely undergo rigorous temporal evaluation for future discovery potential. To address this, we propose BioDisco, a multi-agent framework that draws upon language model-based reasoning and a dual-mode evidence system (biomedical knowledge graphs and automated literature retrieval) for grounded novelty, integrates an internal scoring and feedback loop for iterative refinement, and validates performance through pioneering temporal and human evaluations and a Bradley-Terry paired comparison model to provide statistically-grounded assessment. Our evaluations demonstrate superior novelty and significance over ablated configurations and generalist biomedical agents. Designed for flexibility and modularity, BioDisco allows seamless integration of custom language models or knowledge graphs, and can be run with just a few lines of code.