End-to-End NLP Knowledge Graph Construction

TL;DR

This paper presents SciNLP-KG, an end-to-end framework for extracting multiple relation types from scientific papers to construct a large-scale NLP knowledge graph, aiding automatic leaderboard creation.

Contribution

It introduces novel methods for relation extraction and applies them to 30,000 papers, creating a high-quality, large-scale NLP knowledge graph.

Findings

The KG contains high-quality, reliable information.

The framework successfully extracts four key relation types.

Large-scale construction from 30,000 papers demonstrated effectiveness.

Abstract

This paper studies the end-to-end construction of an NLP Knowledge Graph (KG) from scientific papers. We focus on extracting four types of relations: evaluatedOn between tasks and datasets, evaluatedBy between tasks and evaluation metrics, as well as coreferent and related relations between the same type of entities. For instance, F1-score is coreferent with F-measure. We introduce novel methods for each of these relation types and apply our final framework (SciNLP-KG) to 30,000 NLP papers from ACL Anthology to build a large-scale KG, which can facilitate automatically constructing scientific leaderboards for the NLP community. The results of our experiments indicate that the resulting KG contains high-quality information.