LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model

TL;DR

This paper presents an ALBERT-based model with a novel transfer learning strategy for commonsense validation and explanation tasks, achieving high accuracy and competitive rankings in SemEval-2020 Task 4.

Contribution

We introduce a transfer learning approach between subtasks and enhanced models for improved performance in commonsense validation and explanation.

Findings

Achieved 95.6% accuracy on subtask a

Achieved 94.9% accuracy on subtask b

Ranked 7th and 2nd on leaderboards

Abstract

This paper describes our submission to subtask a and b of SemEval-2020 Task 4. For subtask a, we use a ALBERT based model with improved input form to pick out the common sense statement from two statement candidates. For subtask b, we use a multiple choice model enhanced by hint sentence mechanism to select the reason from given options about why a statement is against common sense. Besides, we propose a novel transfer learning strategy between subtasks which help improve the performance. The accuracy scores of our system are 95.6 / 94.9 on official test set and rank 7$^{th}$ / 2$^{nd}$ on Post-Evaluation leaderboard.