The Automated Verification of Textual Claims (AVeriTeC) Shared Task

TL;DR

The paper presents the AVeriTeC shared task focused on automating the verification of textual claims by retrieving evidence and predicting their veracity, with results indicating significant progress in automated fact-checking.

Contribution

It introduces a shared task framework for automated claim verification, evaluates multiple systems, and highlights key advancements and challenges in the field.

Findings

18 out of 21 submissions surpassed the baseline.

The winning team achieved an AVeriTeC score of 63%.

The shared task provided insights into effective evidence retrieval and veracity prediction methods.

Abstract

The Automated Verification of Textual Claims (AVeriTeC) shared task asks participants to retrieve evidence and predict veracity for real-world claims checked by fact-checkers. Evidence can be found either via a search engine, or via a knowledge store provided by the organisers. Submissions are evaluated using AVeriTeC score, which considers a claim to be accurately verified if and only if both the verdict is correct and retrieved evidence is considered to meet a certain quality threshold. The shared task received 21 submissions, 18 of which surpassed our baseline. The winning team was TUDA_MAI with an AVeriTeC score of 63%. In this paper we describe the shared task, present the full results, and highlight key takeaways from the shared task.