ELEVANT: A Fully Automatic Fine-Grained Entity Linking Evaluation and Analysis Tool

TL;DR

Elevant is an automated tool that evaluates fine-grained entity linking systems, providing detailed error analysis and intuitive visualizations to compare system performance against ground truth benchmarks.

Contribution

It introduces a fully automatic evaluation framework with error categorization and visualization, enhancing analysis of entity linkers without manual intervention.

Findings

Automatic breakdown of errors by categories and entity types

Intuitive visualization of linker performance versus ground truth

Accessible online demo and open-source code

Abstract

We present Elevant, a tool for the fully automatic fine-grained evaluation of a set of entity linkers on a set of benchmarks. Elevant provides an automatic breakdown of the performance by various error categories and by entity type. Elevant also provides a rich and compact, yet very intuitive and self-explanatory visualization of the results of a linker on a benchmark in comparison to the ground truth. A live demo, the link to the complete code base on GitHub and a link to a demo video are provided under https://elevant.cs.uni-freiburg.de .