MuLMS-AZ: An Argumentative Zoning Dataset for the Materials Science Domain

TL;DR

This paper introduces MuLMS-AZ, a new annotated dataset for classifying argumentative zones in materials science research papers, demonstrating the importance of domain-specific models for accurate classification.

Contribution

The work provides a novel materials science-specific AZ dataset with multi-label annotations and shows the effectiveness of domain-specific transformers for AZ classification.

Findings

High inter-annotator agreement achieved

Domain-specific transformers outperform generic models

AZ categories are partially transferable across domains

Abstract

Scientific publications follow conventionalized rhetorical structures. Classifying the Argumentative Zone (AZ), e.g., identifying whether a sentence states a Motivation, a Result or Background information, has been proposed to improve processing of scholarly documents. In this work, we adapt and extend this idea to the domain of materials science research. We present and release a new dataset of 50 manually annotated research articles. The dataset spans seven sub-topics and is annotated with a materials-science focused multi-label annotation scheme for AZ. We detail corpus statistics and demonstrate high inter-annotator agreement. Our computational experiments show that using domain-specific pre-trained transformer-based text encoders is key to high classification performance. We also find that AZ categories from existing datasets in other domains are transferable to varying degrees.