Term Relevance Feedback for Contextual Named Entity Retrieval

TL;DR

This paper investigates how user-provided relevance feedback and importance weighting of context terms enhance the performance of Contextual Named Entity Retrieval systems, demonstrating the value of human input over automated methods.

Contribution

It introduces a user-centric approach to term relevance feedback and importance weighting in CNER, showing improvements over automated techniques.

Findings

User identification of important terms outperforms automated methods.

Indicating relative importance of terms further improves retrieval.

Human feedback significantly enhances CNER system effectiveness.

Abstract

We address the role of a user in Contextual Named Entity Retrieval (CNER), showing (1) that user identification of important context-bearing terms is superior to automated approaches, and (2) that further gains are possible if the user indicates the relative importance of those terms. CNER is similar in spirit to List Question answering and Entity disambiguation. However, the main focus of CNER is to obtain user feedback for constructing a profile for a class of entities on the fly and use that to retrieve entities from free text. Given a sentence, and an entity selected from that sentence, CNER aims to retrieve sentences that have entities similar to query entity. This paper explores obtaining term relevance feedback and importance weighting from humans in order to improve a CNER system. We report our findings based on the efforts of IR researchers as well as crowdsourced workers.