Overview of TREC 2025 Biomedical Generative Retrieval (BioGen) Track

TL;DR

This paper reviews the TREC 2025 Biomedical Generative Retrieval (BioGen) Track, highlighting recent progress in biomedical LLMs, their capabilities, challenges like hallucinations, and evaluation methods for grounded information retrieval.

Contribution

It provides an overview of the BioGen track, emphasizing the evaluation of biomedical LLMs' retrieval and generation capabilities and addressing challenges like hallucinations.

Findings

LLMs show strong biomedical information processing abilities.

Hallucination remains a significant challenge in biomedical LLMs.

Evaluation methods for grounding in biomedical retrieval are discussed.

Abstract

Recent advances in large language models (LLMs) have made significant progress across multiple biomedical tasks, including biomedical question answering, lay-language summarization of the biomedical literature, and clinical note summarization. These models have demonstrated strong capabilities in processing and synthesizing complex biomedical information and in generating fluent, human-like responses. Despite these advancements, hallucinations or confabulations remain key challenges when using LLMs in biomedical and other high-stakes domains. Inaccuracies may be particularly harmful in high-risk situations, such as medical question answering, making clinical decisions, or appraising biomedical research. Studies on the evaluation of the LLMs' abilities to ground generated statements in verifiable sources have shown that models perform significantly