heiDS at ArchEHR-QA 2025: From Fixed-k to Query-dependent-k for Retrieval Augmented Generation

TL;DR

This paper introduces a query-dependent retrieval strategy for RAG frameworks in clinical question answering, improving factual accuracy over fixed-k methods by employing novel truncation techniques.

Contribution

It proposes new query-dependent-k retrieval methods, including autocut* and elbow, enhancing answer relevance and factuality in clinical RAG systems.

Findings

Query-dependent-k strategies outperform fixed-k in accuracy.

Autocut* and elbow methods improve retrieval relevance.

Enhanced factual correctness in clinical answer generation.

Abstract

This paper presents the approach of our team called heiDS for the ArchEHR-QA 2025 shared task. A pipeline using a retrieval augmented generation (RAG) framework is designed to generate answers that are attributed to clinical evidence from the electronic health records (EHRs) of patients in response to patient-specific questions. We explored various components of a RAG framework, focusing on ranked list truncation (RLT) retrieval strategies and attribution approaches. Instead of using a fixed top-k RLT retrieval strategy, we employ a query-dependent-k retrieval strategy, including the existing surprise and autocut methods and two new methods proposed in this work, autocut* and elbow. The experimental results show the benefits of our strategy in producing factual and relevant answers when compared to a fixed-$k$.