PcMSP: A Dataset for Scientific Action Graphs Extraction from Polycrystalline Materials Synthesis Procedure Text

TL;DR

This paper introduces PcMSP, a new high-quality dataset of annotated scientific texts on polycrystalline materials synthesis, enabling improved information extraction and natural language processing tasks in materials science.

Contribution

It provides the first annotated dataset for synthesis procedures in materials science, along with a comprehensive annotation scheme and baseline NLP task evaluations.

Findings

State-of-the-art models perform well but still have significant room for improvement.

High inter-annotator agreement indicates dataset quality.

Error analysis reveals unique challenges in scientific text extraction.

Abstract

Scientific action graphs extraction from materials synthesis procedures is important for reproducible research, machine automation, and material prediction. But the lack of annotated data has hindered progress in this field. We demonstrate an effort to annotate Polycrystalline Materials Synthesis Procedures (PcMSP) from 305 open access scientific articles for the construction of synthesis action graphs. This is a new dataset for material science information extraction that simultaneously contains the synthesis sentences extracted from the experimental paragraphs, as well as the entity mentions and intra-sentence relations. A two-step human annotation and inter-annotator agreement study guarantee the high quality of the PcMSP corpus. We introduce four natural language processing tasks: sentence classification, named entity recognition, relation classification, and joint extraction of entities and relations. Comprehensive experiments validate the effectiveness of several state-of-the-art models for these challenges while leaving large space for improvement. We also perform the error analysis and point out some unique challenges that require further investigation. We will release our annotation scheme, the corpus, and codes to the research community to alleviate the scarcity of labeled data in this domain.