Can Hallucinations Help? Boosting LLMs for Drug Discovery

TL;DR

This paper explores how hallucinations in large language models can be harnessed to improve molecule property prediction in drug discovery, challenging the view that hallucinations are solely problematic.

Contribution

It demonstrates that hallucinations can enhance predictive accuracy in drug discovery tasks and identifies the types of hallucinations that are most beneficial.

Findings

Hallucinations significantly improve model accuracy in some cases.

Larger models benefit more from hallucinations.

Structural misdescriptions are the most impactful hallucination type.

Abstract

Hallucinations in large language models (LLMs), plausible but factually inaccurate text, are often viewed as undesirable. However, recent work suggests that such outputs may hold creative potential. In this paper, we investigate whether hallucinations can improve LLMs on molecule property prediction, a key task in early-stage drug discovery. We prompt LLMs to generate natural language descriptions from molecular SMILES strings and incorporate these often hallucinated descriptions into downstream classification tasks. Evaluating seven instruction-tuned LLMs across five datasets, we find that hallucinations significantly improve predictive accuracy for some models. Notably, Falcon3-Mamba-7B outperforms all baselines when hallucinated text is included, while hallucinations generated by GPT-4o consistently yield the greatest gains between models. We further identify and categorize over 18,000 beneficial hallucinations, with structural misdescriptions emerging as the most impactful type, suggesting that hallucinated statements about molecular structure may increase model confidence. Ablation studies show that larger models benefit more from hallucinations, while temperature has a limited effect. Our findings challenge conventional views of hallucination as purely problematic and suggest new directions for leveraging hallucinations as a useful signal in scientific modeling tasks like drug discovery.