Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
Alberto Barron-Cedeno, Tamer Elsayed, Preslav Nakov, Giovanni Da San, Martino, Maram Hasanain, Reem Suwaileh, Fatima Haouari, Nikolay Babulkov,, Bayan Hamdan, Alex Nikolov, Shaden Shaar, and Zien Sheikh Ali

TL;DR
The paper provides an overview of the CheckThat! 2020 lab, detailing five tasks related to claim verification in social media across English and Arabic, highlighting participant approaches, results, and released datasets for future research.
Contribution
It introduces a comprehensive benchmark for claim verification tasks, reports on participant methods and results, and releases datasets and evaluation scripts to advance research in social media claim verification.
Findings
Deep neural networks outperformed baselines across tasks.
Increased participation compared to previous year.
Sizable improvements achieved on all tasks.
Abstract
We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the full pipeline of claim verification in social media: Task 1 on check-worthiness estimation, Task 2 on retrieving previously fact-checked claims, Task 3 on evidence retrieval, and Task 4 on claim verification. The lab is completed with Task 5 on check-worthiness estimation in political debates and speeches. A total of 67 teams registered to participate in the lab (up from 47 at CLEF 2019), and 23 of them actually submitted runs (compared to 14 at CLEF 2019). Most teams used deep neural networks based on BERT, LSTMs, or CNNs, and achieved sizable improvements over the baselines on all tasks. Here we describe the tasks setup, the evaluation results, and a summary of the approaches used by the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsLinear Layer · Attention Dropout · Adam · Dense Connections · Linear Warmup With Linear Decay · Residual Connection · Dropout · Layer Normalization · Refunds@Expedia|||How do I get a full refund from Expedia? · Multi-Head Attention
