BioPIE: A Biomedical Protocol Information Extraction Dataset for High-Reasoning-Complexity Experiment Question Answer

TL;DR

BioPIE is a new dataset designed for extracting detailed, procedure-centric knowledge graphs from biomedical protocols, enabling advanced reasoning in biomedical question answering systems.

Contribution

The paper introduces BioPIE, a fine-grained biomedical dataset supporting high-reasoning and multi-step questions, filling a gap in existing biomedical knowledge resources.

Findings

Information extraction methods improved on BioPIE

QA system leveraging BioPIE shows performance gains

Supports complex biomedical experimental reasoning

Abstract

Question Answer (QA) systems for biomedical experiments facilitate cross-disciplinary communication, and serve as a foundation for downstream tasks, e.g., laboratory automation. High Information Density (HID) and Multi-Step Reasoning (MSR) pose unique challenges for biomedical experimental QA. While extracting structured knowledge, e.g., Knowledge Graphs (KGs), can substantially benefit biomedical experimental QA. Existing biomedical datasets focus on general or coarsegrained knowledge and thus fail to support the fine-grained experimental reasoning demanded by HID and MSR. To address this gap, we introduce Biomedical Protocol Information Extraction Dataset (BioPIE), a dataset that provides procedure-centric KGs of experimental entities, actions, and relations at a scale that supports reasoning over biomedical experiments across protocols. We evaluate information extraction methods on BioPIE, and implement a QA system that leverages BioPIE, showcasing performance gains on test, HID, and MSR question sets, showing that the structured experimental knowledge in BioPIE underpins both AI-assisted and more autonomous biomedical experimentation.