SemIRNet: A Semantic Irony Recognition Network for Multimodal Sarcasm Detection

TL;DR

SemIRNet is a novel multimodal sarcasm detection model that integrates ConceptNet knowledge, cross-modal semantic similarity modules, and contrastive learning to improve accuracy and robustness.

Contribution

The paper introduces SemIRNet, which uniquely combines ConceptNet, multi-level semantic similarity detection, and contrastive loss for enhanced irony recognition.

Findings

Achieved 88.87% accuracy and 86.33% F1 score, outperforming existing methods.

Demonstrated the effectiveness of knowledge fusion and semantic similarity modules.

Validated improvements through ablation experiments.

Abstract

Aiming at the problem of difficulty in accurately identifying graphical implicit correlations in multimodal irony detection tasks, this paper proposes a Semantic Irony Recognition Network (SemIRNet). The model contains three main innovations: (1) The ConceptNet knowledge base is introduced for the first time to acquire conceptual knowledge, which enhances the model's common-sense reasoning ability; (2) Two cross-modal semantic similarity detection modules at the word level and sample level are designed to model graphic-textual correlations at different granularities; and (3) A contrastive learning loss function is introduced to optimize the spatial distribution of the sample features, which improves the separability of positive and negative samples. Experiments on a publicly available multimodal irony detection benchmark dataset show that the accuracy and F1 value of this model are improved by 1.64% and 2.88% to 88.87% and 86.33%, respectively, compared with the existing optimal methods. Further ablation experiments verify the important role of knowledge fusion and semantic similarity detection in improving the model performance.