Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions

TL;DR

This paper introduces a DeBERTa-based system augmented with LLM-generated synthetic data to classify political question evasions, achieving competitive results in SemEval-2026 Task 6.

Contribution

The novel integration of LLM-based data augmentation with DeBERTa enhances minority class detection in political discourse classification tasks.

Findings

Achieved a Macro F1 of 0.76 on Task 1, ranking 8th out of 40.

LLM-based augmentation improved minority class recall.

Error analysis shows confusion between Ambivalent and Clear Reply responses.

Abstract

This paper presents the Duluth approach to SemEval-2026 Task 6 on CLARITY: Unmasking Political Question Evasions. We address Task 1 (clarity-level classification) and Task 2 (evasion-level classification), both of which involve classifying question--answer pairs from U.S.\ presidential interviews using a two-level taxonomy of response clarity. Our system is based on DeBERTa-V3-base, extended with focal loss, layer-wise learning rate decay, and boolean discourse features. To address class imbalance in the training data, we augment minority classes using synthetic examples generated by Gemini 3 and Claude Sonnet 4.5. Our best configuration achieved a Macro F1 of 0.76 on the Task 1 evaluation set, placing 8th out of 40 teams. The top-ranked system (TeleAI) achieved 0.89, while the mean score across participants was 0.70. Error analysis reveals that the dominant source of misclassification is confusion between Ambivalent and Clear Reply responses, a pattern that mirrors disagreements among human annotators. Our findings demonstrate that LLM-based data augmentation can meaningfully improve minority-class recall on nuanced political discourse tasks.