SemEval-2020 Task 7: Assessing Humor in Edited News Headlines

TL;DR

This paper presents the SemEval-2020 shared task focused on quantifying and comparing humor in edited news headlines, involving crowdsourced ratings and competitive participation.

Contribution

It introduces a new dataset and evaluation framework for assessing humor in news headlines, fostering research in computational humor detection and comparison.

Findings

48 teams participated in humor rating subtask

31 teams participated in humor comparison subtask

The task is the most popular shared computational humor challenge to date

Abstract

This paper describes the SemEval-2020 shared task "Assessing Humor in Edited News Headlines." The task's dataset contains news headlines in which short edits were applied to make them funny, and the funniness of these edited headlines was rated using crowdsourcing. This task includes two subtasks, the first of which is to estimate the funniness of headlines on a humor scale in the interval 0-3. The second subtask is to predict, for a pair of edited versions of the same original headline, which is the funnier version. To date, this task is the most popular shared computational humor task, attracting 48 teams for the first subtask and 31 teams for the second.