Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentation

TL;DR

This paper presents a deep learning approach with contextually sensitive lexical data augmentation for identifying and ranking interesting claims on social media across multiple languages, achieving top results in the CLEF2021 CheckThat! challenge.

Contribution

It introduces a novel lexical augmentation technique for transformer models that enhances claim classification and ranking across diverse languages in social media data.

Findings

Augmentation improved performance across all languages.

Best system achieved top results for Arabic.

Performance scaled with training data quantity.

Abstract

This paper discusses the approach used by the Accenture Team for CLEF2021 CheckThat! Lab, Task 1, to identify whether a claim made in social media would be interesting to a wide audience and should be fact-checked. Twitter training and test data were provided in English, Arabic, Spanish, Turkish, and Bulgarian. Claims were to be classified (check-worthy/not check-worthy) and ranked in priority order for the fact-checker. Our method used deep neural network transformer models with contextually sensitive lexical augmentation applied on the supplied training datasets to create additional training samples. This augmentation approach improved the performance for all languages. Overall, our architecture and data augmentation pipeline produced the best submitted system for Arabic, and performance scales according to the quantity of provided training data for English, Spanish, Turkish, and Bulgarian. This paper investigates the deep neural network architectures for each language as well as the provided data to examine why the approach worked so effectively for Arabic, and discusses additional data augmentation measures that should could be useful to this problem.