ANA at SemEval-2020 Task 4: mUlti-task learNIng for cOmmonsense reasoNing (UNION)

TL;DR

This paper introduces UNION, a multi-task learning framework for commonsense reasoning that leverages multiple datasets to generate meaningful explanations for nonsensical statements, outperforming baseline models.

Contribution

The paper presents UNION, a novel end-to-end multi-task learning approach that improves commonsense reasoning explanations by integrating various datasets and new automatic evaluation metrics.

Findings

Outperforms competitors with a human evaluation score of 2.10

Achieves a BLEU score of 15.7

Demonstrates improved explanation quality over simple fine-tuning methods

Abstract

In this paper, we describe our mUlti-task learNIng for cOmmonsense reasoNing (UNION) system submitted for Task C of the SemEval2020 Task 4, which is to generate a reason explaining why a given false statement is non-sensical. However, we found in the early experiments that simple adaptations such as fine-tuning GPT2 often yield dull and non-informative generations (e.g. simple negations). In order to generate more meaningful explanations, we propose UNION, a unified end-to-end framework, to utilize several existing commonsense datasets so that it allows a model to learn more dynamics under the scope of commonsense reasoning. In order to perform model selection efficiently, accurately and promptly, we also propose a couple of auxiliary automatic evaluation metrics so that we can extensively compare the models from different perspectives. Our submitted system not only results in a good performance in the proposed metrics but also outperforms its competitors with the highest achieved score of 2.10 for human evaluation while remaining a BLEU score of 15.7. Our code is made publicly available at GitHub.