Can Pre-trained Language Models Understand Chinese Humor?

TL;DR

This paper systematically investigates whether pre-trained language models can understand Chinese humor, using a comprehensive evaluation framework and dataset, providing insights for future improvements in humor-related NLP tasks.

Contribution

First to systematically evaluate the humor understanding ability of PLMs in Chinese using a new dataset and a multi-step framework.

Findings

PLMs show limited humor understanding capabilities.

The evaluation framework reveals specific strengths and weaknesses of PLMs in humor tasks.

Insights guide future optimization of PLMs for humor comprehension.

Abstract

Humor understanding is an important and challenging research in natural language processing. As the popularity of pre-trained language models (PLMs), some recent work makes preliminary attempts to adopt PLMs for humor recognition and generation. However, these simple attempts do not substantially answer the question: {\em whether PLMs are capable of humor understanding?} This paper is the first work that systematically investigates the humor understanding ability of PLMs. For this purpose, a comprehensive framework with three evaluation steps and four evaluation tasks is designed. We also construct a comprehensive Chinese humor dataset, which can fully meet all the data requirements of the proposed evaluation framework. Our empirical study on the Chinese humor dataset yields some valuable observations, which are of great guiding value for future optimization of PLMs in humor understanding and generation.