Integrating curation into scientific publishing to train AI models

TL;DR

This paper introduces a new dataset called SourceData-NLP, created by integrating data curation into the scientific publishing process to improve AI training for biomedical research.

Contribution

The novel integration of curation into publishing enables comprehensive annotation of experimental roles and methodologies alongside bioentity recognition.

Findings

SourceData-NLP contains over 620,000 annotated biomedical entities from 3,223 articles.

The dataset supports AI training for tasks like named-entity recognition and figure caption segmentation.

A new context-dependent semantic task was introduced to assess entity roles in experiments.

Abstract

High-throughput extraction and structured labeling of data from academic articles are crucial for enabling downstream machine learning applications and secondary analyses. Current approaches lack integration with the publishing process and comprehensive annotation of experimental roles and methodologies alongside bioentity recognition.

We embedded multimodal data curation into the academic publishing process to annotate segmented figure panels and captions, combining natural language processing with authors’ feedback to increase annotation accuracy. The resulting dataset, SourceData-NLP, comprises over 620 000 annotated biomedical entities, curated from 18 689 figures in 3223 articles in molecular and cell biology. Annotations include eight classes of bioentities (small molecules, gene products, subcellular components, cell lines, cell types, tissues, organisms, and diseases), plus additional classes that delineate the entities’ roles in experimental designs and methodologies. We evaluate the utility of the dataset for training AI models using named-entity recognition, segmentation of figure captions into their constituent panels, and a novel context-dependent semantic task that assesses whether an entity is a controlled intervention target or a measurement object. We also demonstrate multi-modal applications for segmenting figures into panel images and their corresponding captions.

Trained models are available at https://huggingface.co/EMBO. The SourceData-NLP dataset and code are available at https://github.com/source-data/soda-data, https://github.com/source-data/soda-model, and https://github.com/source-data/soda_image_segmentation.

Figures