Not All Pretraining are Created Equal: Threshold Tuning and Class Weighting for Imbalanced Polarization Tasks in Low-Resource Settings

TL;DR

This paper presents Transformer-based multilingual systems with class-weighted loss and threshold tuning to improve polarization detection in social media, especially in low-resource languages like Swahili.

Contribution

It introduces a novel combination of multilingual models, class weighting, and threshold tuning specifically for imbalanced polarization tasks in low-resource settings.

Findings

Achieved 0.8032 macro-F1 on binary detection

Demonstrated effectiveness of class-weighted loss and threshold tuning

Identified challenges with implicit polarization and code-switching

Abstract

This paper describes my submission to the Polarization Shared Task at SemEval-2025, which addresses polarization detection and classification in social media text. I develop Transformer-based systems for English and Swahili across three subtasks: binary polarization detection, multi-label target type classification, and multi-label manifestation identification. The approach leverages multilingual and African language-specialized models (mDeBERTa-v3-base, SwahBERT, AfriBERTa-large), class-weighted loss functions, iterative stratified data splitting, and per-label threshold tuning to handle severe class imbalance. The best configuration, mDeBERTa-v3-base, achieves 0.8032 macro-F1 on validation for binary detection, with competitive performance on multi-label tasks (up to 0.556 macro-F1). Error analysis reveals persistent challenges with implicit polarization, code-switching, and distinguishing heated political discourse from genuine polarization.