Discovering Entities with Just a Little Help from You

TL;DR

This paper introduces a human-in-the-loop approach for entity linking that improves disambiguation of long-tail and emerging entities by involving user feedback to generate better candidate suggestions, especially when automated methods fall short.

Contribution

It presents novel gradient interleaving retrieval methods that incorporate user feedback to enhance candidate generation for entity linking, addressing the challenge of long-tail and emerging entities.

Findings

Outperforms baseline methods in intrinsic and extrinsic evaluations.

Engages users effectively in the disambiguation process.

Improves linking accuracy for long-tail and emerging entities.

Abstract

Linking entities like people, organizations, books, music groups and their songs in text to knowledge bases (KBs) is a fundamental task for many downstream search and mining applications. Achieving high disambiguation accuracy crucially depends on a rich and holistic representation of the entities in the KB. For popular entities, such a representation can be easily mined from Wikipedia, and many current entity disambiguation and linking methods make use of this fact. However, Wikipedia does not contain long-tail entities that only few people are interested in, and also at times lags behind until newly emerging entities are added. For such entities, mining a suitable representation in a fully automated fashion is very difficult, resulting in poor linking accuracy. What can automatically be mined, though, is a high-quality representation given the context of a new entity occurring in any text. Due to the lack of knowledge about the entity, no method can retrieve these occurrences automatically with high precision, resulting in a chicken-egg problem. To address this, our approach automatically generates candidate occurrences of entities, prompting the user for feedback to decide if the occurrence refers to the actual entity in question. This feedback gradually improves the knowledge and allows our methods to provide better candidate suggestions to keep the user engaged. We propose novel human-in-the-loop retrieval methods for generating candidates based on gradient interleaving of diversification and textual relevance approaches. We conducted extensive experiments on the FACC dataset, showing that our approaches convincingly outperform carefully selected baselines in both intrinsic and extrinsic measures while keeping users engaged.