PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Transformers for Patronizing and Condescending Language Detection

TL;DR

This paper presents a novel Transformer-based approach with ensemble methods for detecting patronizing and condescending language, achieving top rankings in SemEval-2022 Task 4.

Contribution

It introduces a discriminative fine-tuning strategy for Transformers to improve subtle language detection in PCL, a novel approach in this domain.

Findings

Achieved 1st place in Subtask 1

Achieved 5th place in Subtask 2

Demonstrated effectiveness of fine-tuning strategies

Abstract

Patronizing and condescending language (PCL) has a large harmful impact and is difficult to detect, both for human judges and existing NLP systems. At SemEval-2022 Task 4, we propose a novel Transformer-based model and its ensembles to accurately understand such language context for PCL detection. To facilitate comprehension of the subtle and subjective nature of PCL, two fine-tuning strategies are applied to capture discriminative features from diverse linguistic behaviour and categorical distribution. The system achieves remarkable results on the official ranking, including 1st in Subtask 1 and 5th in Subtask 2. Extensive experiments on the task demonstrate the effectiveness of our system and its strategies.