GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases

TL;DR

GeneAgent is a novel language agent that autonomously interacts with biological databases to improve gene set knowledge discovery, significantly reducing hallucinations and providing more reliable insights compared to standard LLMs.

Contribution

It introduces a self-verification language agent for gene set analysis that outperforms GPT-4 and minimizes hallucinations through domain knowledge integration.

Findings

GeneAgent outperforms GPT-4 on benchmark gene sets.

Self-verification reduces hallucinations in gene analysis.

Application to mouse melanoma gene sets yields novel insights.

Abstract

Gene set knowledge discovery is essential for advancing human functional genomics. Recent studies have shown promising performance by harnessing the power of Large Language Models (LLMs) on this task. Nonetheless, their results are subject to several limitations common in LLMs such as hallucinations. In response, we present GeneAgent, a first-of-its-kind language agent featuring self-verification capability. It autonomously interacts with various biological databases and leverages relevant domain knowledge to improve accuracy and reduce hallucination occurrences. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms standard GPT-4 by a significant margin. Moreover, a detailed manual review confirms the effectiveness of the self-verification module in minimizing hallucinations and generating more reliable analytical narratives. To demonstrate its practical utility, we apply GeneAgent to seven novel gene sets derived from mouse B2905 melanoma cell lines, with expert evaluations showing that GeneAgent offers novel insights into gene functions and subsequently expedites knowledge discovery.