Mao-Zedong At SemEval-2023 Task 4: Label Represention Multi-Head Attention Model With Contrastive Learning-Enhanced Nearest Neighbor Mechanism For Multi-Label Text Classification

TL;DR

This paper introduces a novel multi-label text classification method combining label-specific multi-head attention with contrastive learning-enhanced nearest neighbor retrieval, achieving competitive results on SemEval-2023 Task 4.

Contribution

It proposes a new model integrating label-aware multi-head attention and contrastive learning with K-nearest neighbors for improved human value classification.

Findings

Achieved an F1 score of 0.533 on the test set.

Ranked fourth on the SemEval-2023 leaderboard.

Demonstrated effectiveness of contrastive learning in multi-label classification.

Abstract

The study of human values is essential in both practical and theoretical domains. With the development of computational linguistics, the creation of large-scale datasets has made it possible to automatically recognize human values accurately. SemEval 2023 Task 4\cite{kiesel:2023} provides a set of arguments and 20 types of human values that are implicitly expressed in each argument. In this paper, we present our team's solution. We use the Roberta\cite{liu_roberta_2019} model to obtain the word vector encoding of the document and propose a multi-head attention mechanism to establish connections between specific labels and semantic components. Furthermore, we use a contrastive learning-enhanced K-nearest neighbor mechanism\cite{su_contrastive_2022} to leverage existing instance information for prediction. Our approach achieved an F1 score of 0.533 on the test set and ranked fourth on the leaderboard.