CollabKG: A Learnable Human-Machine-Cooperative Information Extraction Toolkit for (Event) Knowledge Graph Construction

TL;DR

CollabKG is a versatile, learnable human-machine-cooperative toolkit that unifies multiple information extraction tasks for knowledge graph construction, improving efficiency, quality, and adaptability through advanced prompting and interactive learning.

Contribution

The paper introduces CollabKG, a novel IE toolkit supporting multi-task extraction, human-machine cooperation, and self-learning for knowledge graph and event graph construction.

Findings

Significantly improves annotation quality and efficiency.

Supports multi-task IE including NER, RE, and EE.

Demonstrates high stability and customization capabilities.

Abstract

In order to construct or extend entity-centric and event-centric knowledge graphs (KG and EKG), the information extraction (IE) annotation toolkit is essential. However, existing IE toolkits have several non-trivial problems, such as not supporting multi-tasks, not supporting automatic updates. In this work, we present CollabKG, a learnable human-machine-cooperative IE toolkit for KG and EKG construction. Specifically, for the multi-task issue, CollabKG unifies different IE subtasks, including named entity recognition (NER), entity-relation triple extraction (RE), and event extraction (EE), and supports both KG and EKG. Then, combining advanced prompting-based IE technology, the human-machine-cooperation mechanism with LLMs as the assistant machine is presented which can provide a lower cost as well as a higher performance. Lastly, owing to the two-way interaction between the human and machine, CollabKG with learning ability allows self-renewal. Besides, CollabKG has several appealing features (e.g., customization, training-free, propagation, etc.) that make the system powerful, easy-to-use, and high-productivity. We holistically compare our toolkit with other existing tools on these features. Human evaluation quantitatively illustrates that CollabKG significantly improves annotation quality, efficiency, and stability simultaneously.