When in Doubt, Consult: Expert Debate for Sexism Detection via Confidence-Based Routing

TL;DR

This paper introduces a two-stage framework for detecting subtle online sexism, combining stabilized training with a dynamic expert routing mechanism to improve accuracy on ambiguous cases.

Contribution

It proposes a novel two-stage approach with targeted regularization and a reasoning-based expert system for better sexism detection in complex, noisy datasets.

Findings

Outperforms existing methods on public benchmarks.

Achieves significant F1 score improvements (+4.48%, +1.30%).

Enhances detection of subtle, context-dependent sexism.

Abstract

Online sexism increasingly appears in subtle, context-dependent forms that evade traditional detection methods. Its interpretation often depends on overlapping linguistic, psychological, legal, and cultural dimensions, which produce mixed and sometimes contradictory signals in annotated datasets. These inconsistencies, combined with label scarcity and class imbalance, result in unstable decision boundaries and cause fine-tuned models to overlook subtler, underrepresented forms of harm. To address these challenges, we propose a two-stage framework that unifies (i) targeted training procedures to better regularize supervision to scarce and noisy data with (ii) selective, reasoning-based inference to handle ambiguous or borderline cases. First, we stabilize the training combining class-balanced focal loss, class-aware batching, and post-hoc threshold calibration, strategies for the firs time adapted for this domain to mitigate label imbalance and noisy supervision. Second, we bridge the gap between efficiency and reasoning with a a dynamic routing mechanism that distinguishes between unambiguous instances and complex cases requiring a deliberative process. This reasoning process results in the novel Collaborative Expert Judgment (CEJ) module which prompts multiple personas and consolidates their reasoning through a judge model. Our approach outperforms existing approaches across several public benchmarks, with F1 gains of +4.48% and +1.30% on EDOS Tasks A and B, respectively, and a +2.79% improvement in ICM on EXIST 2025 Task 1.1.