Benchmarking Named Entity Disambiguation approaches for Streaming Graphs

TL;DR

This paper benchmarks two state-of-the-art named entity disambiguation systems, AIDA and MENED, on streaming data, highlighting their approaches, experimental setup, and comparative performance in real-world scenarios.

Contribution

It provides a comparative analysis of graph-based and statistical inference NED approaches using a standardized dataset and evaluation framework.

Findings

AIDA outperforms MENED in accuracy on the benchmark dataset.

Graph-based approach shows better scalability for streaming data.

Statistical approach offers advantages in ambiguous cases.

Abstract

Named Entity Disambiaguation (NED) is a central task for applications dealing with natural language text. Assume that we have a graph based knowledge base (subsequently referred as Knowledge Graph) where nodes represent various real world entities such as people, location, organization and concepts. Given data sources such as social media streams and web pages Entity Linking is the task of mapping named entities that are extracted from the data to those present in the Knowledge Graph. This is an inherently difficult task due to several reasons. Almost all these data sources are generated without any formal ontology; the unstructured nature of the input, limited context and the ambiguity involved when multiple entities are mapped to the same name make this a hard task. This report looks at two state of the art systems employing two distinctive approaches: graph based Accurate Online Disambiguation of Entities (AIDA) and Mined Evidence Named Entity Disambiguation (MENED), which employs a statistical inference approach. We compare both approaches using the data set and queries provided by the Knowledge Base Population (KBP) track at 2011 NIST Text Analytics Conference (TAC). This report begins with an overview of the respective approaches, followed by detailed description of the experimental setup. It concludes with our findings from the benchmarking exercise.