Mario at EXIST 2025: A Simple Gateway to Effective Multilingual Sexism Detection

TL;DR

This paper introduces a simple, efficient multilingual sexism detection method using hierarchical LoRA adaptation of Llama 3.1, achieving strong performance with minimal parameters and training time across English and Spanish tweets.

Contribution

It presents a novel hierarchical LoRA adaptation with conditional routing for multilingual sexism detection, reducing complexity and resource requirements while improving cross-lingual transfer.

Findings

Achieved 1.7-2.4% F1 improvements through cross-lingual transfer.

Reduced training time by 75% and storage by 98% compared to full fine-tuning.

Maintained competitive performance across all subtasks with minimal parameters.

Abstract

This paper presents our approach to EXIST 2025 Task 1, addressing text-based sexism detection in English and Spanish tweets through hierarchical Low-Rank Adaptation (LoRA) of Llama 3.1 8B. Our method introduces conditional adapter routing that explicitly models label dependencies across three hierarchically structured subtasks: binary sexism identification, source intention detection, and multilabel sexism categorization. Unlike conventional LoRA applications that target only attention layers, we apply adaptation to all linear transformations, enhancing the model's capacity to capture task-specific patterns. In contrast to complex data processing and ensemble approaches, we show that straightforward parameter-efficient fine-tuning achieves strong performance. We train separate LoRA adapters (rank=16, QLoRA 4-bit) for each subtask using unified multilingual training that leverages Llama 3.1's native bilingual capabilities. The method requires minimal preprocessing and uses standard supervised learning. Our multilingual training strategy eliminates the need for separate language-specific models, achieving 1.7-2.4\% F1 improvements through cross-lingual transfer. With only 1.67\% trainable parameters compared to full fine-tuning, our approach reduces training time by 75\% and model storage by 98\%, while achieving competitive performance across all subtasks (ICM-Hard: 0.6774 for binary classification, 0.4991 for intention detection, 0.6519 for multilabel categorization).