ZeroKBC: A Comprehensive Benchmark for Zero-Shot Knowledge Base Completion
Pei Chen, Wenlin Yao, Hongming Zhang, Xiaoman Pan, Dian Yu, Dong Yu,, and Jianshu Chen

TL;DR
This paper introduces ZeroKBC, a comprehensive benchmark for zero-shot knowledge base completion, highlighting the challenges and gaps in current systems when dealing with unseen entities and relations.
Contribution
The paper systematically defines zero-shot KBC scenarios, develops a diverse benchmark, and evaluates existing systems, revealing their limitations and guiding future research directions.
Findings
Current KBC systems perform poorly on ZeroKBC.
ZeroKBC covers diverse zero-shot scenarios.
Analysis suggests directions for improving zero-shot KBC.
Abstract
Knowledge base completion (KBC) aims to predict the missing links in knowledge graphs. Previous KBC tasks and approaches mainly focus on the setting where all test entities and relations have appeared in the training set. However, there has been limited research on the zero-shot KBC settings, where we need to deal with unseen entities and relations that emerge in a constantly growing knowledge base. In this work, we systematically examine different possible scenarios of zero-shot KBC and develop a comprehensive benchmark, ZeroKBC, that covers these scenarios with diverse types of knowledge sources. Our systematic analysis reveals several missing yet important zero-shot KBC settings. Experimental results show that canonical and state-of-the-art KBC systems cannot achieve satisfactory performance on this challenging benchmark. By analyzing the strength and weaknesses of these systems on…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Topic Modeling · Data Quality and Management
MethodsTest · Balanced Selection
