Into the Single Cell Multiverse: an End-to-End Dataset for Procedural Knowledge Extraction in Biomedical Texts

TL;DR

This paper introduces FlaMBé, a comprehensive dataset for extracting procedural knowledge from biomedical texts, especially in single cell research, to improve NLP models and enhance reproducibility in biomedical workflows.

Contribution

The paper presents FlaMBé, the largest expert-curated dataset for procedural knowledge and entity recognition in biomedical texts, addressing a critical gap in structured datasets for this task.

Findings

FlaMBé enables better extraction of workflows from biomedical literature.

The dataset improves named entity recognition and disambiguation for tissue and cell types.

Automating workflow mining can enhance reproducibility in biomedical research.

Abstract

Many of the most commonly explored natural language processing (NLP) information extraction tasks can be thought of as evaluations of declarative knowledge, or fact-based information extraction. Procedural knowledge extraction, i.e., breaking down a described process into a series of steps, has received much less attention, perhaps in part due to the lack of structured datasets that capture the knowledge extraction process from end-to-end. To address this unmet need, we present FlaMB\'e (Flow annotations for Multiverse Biological entities), a collection of expert-curated datasets across a series of complementary tasks that capture procedural knowledge in biomedical texts. This dataset is inspired by the observation that one ubiquitous source of procedural knowledge that is described as unstructured text is within academic papers describing their methodology. The workflows annotated in FlaMB\'e are from texts in the burgeoning field of single cell research, a research area that has become notorious for the number of software tools and complexity of workflows used. Additionally, FlaMB\'e provides, to our knowledge, the largest manually curated named entity recognition (NER) and disambiguation (NED) datasets for tissue/cell type, a fundamental biological entity that is critical for knowledge extraction in the biomedical research domain. Beyond providing a valuable dataset to enable further development of NLP models for procedural knowledge extraction, automating the process of workflow mining also has important implications for advancing reproducibility in biomedical research.