Beyond SMILES: Evaluating Agentic Systems for Drug Discovery

TL;DR

This paper evaluates the generalization and limitations of agentic systems in drug discovery, highlighting capability gaps and proposing design requirements for future frameworks to serve as effective computational partners.

Contribution

It systematically assesses six agentic frameworks across diverse drug discovery tasks, identifying key capability gaps and proposing a capability matrix for future development.

Findings

Five major capability gaps identified in current frameworks.

Frontier LLMs can reason about peptides as well as small molecules.

Most frameworks do not leverage the full potential of advanced LLMs.

Abstract

Agentic systems for drug discovery have demonstrated autonomous synthesis planning, literature mining, and molecular design. We ask how well they generalize. Evaluating six frameworks against 15 task classes drawn from peptide therapeutics, in vivo pharmacology, and resource-constrained settings, we find five capability gaps: no support for protein language models or peptide-specific prediction, no bridges between in vivo and in silico data, reliance on LLM inference with no pathway to ML training or reinforcement learning, assumptions tied to large-pharma resources, and single-objective optimization that ignores safety-efficacy-stability trade-offs. A paired knowledge-probing experiment suggests the bottleneck is architectural rather than epistemic: four frontier LLMs reason about peptides at levels comparable to small molecules, yet no framework exposes this capability. We propose design requirements and a capability matrix for next-generation frameworks that function as computational partners under realistic constraints.