Find Parent then Label Children: A Two-stage Taxonomy Completion Method with Pre-trained Language Model

TL;DR

This paper introduces ATTEMPT, a two-stage taxonomy completion method that leverages pre-trained language models to accurately insert new concepts into hierarchical taxonomies, outperforming existing approaches.

Contribution

The paper presents a novel two-stage approach that finds parent nodes and labels children for taxonomy updates, effectively utilizing pre-trained language models for hypernym recognition.

Findings

ATTEMPT outperforms existing methods on multiple datasets.

The approach effectively updates and extends taxonomies.

Pre-trained language models enhance hypernym/hyponymy recognition.

Abstract

Taxonomies, which organize domain concepts into hierarchical structures, are crucial for building knowledge systems and downstream applications. As domain knowledge evolves, taxonomies need to be continuously updated to include new concepts. Previous approaches have mainly focused on adding concepts to the leaf nodes of the existing hierarchical tree, which does not fully utilize the taxonomy's knowledge and is unable to update the original taxonomy structure (usually involving non-leaf nodes). In this paper, we propose a two-stage method called ATTEMPT for taxonomy completion. Our method inserts new concepts into the correct position by finding a parent node and labeling child nodes. Specifically, by combining local nodes with prompts to generate natural sentences, we take advantage of pre-trained language models for hypernym/hyponymy recognition. Experimental results on two public datasets (including six domains) show that ATTEMPT performs best on both taxonomy completion and extension tasks, surpassing existing methods.