Optimizing What We Trust: Reliability-Guided QUBO Selection of Multi-Agent Weak Framing Signals for Arabic Sentiment Prediction

TL;DR

This paper introduces a reliability-guided approach for selecting high-quality data subsets in Arabic sentiment analysis, improving robustness and transferability by focusing on data curation rather than label aggregation.

Contribution

It presents a novel multi-agent LLM framework that estimates instance reliability and guides QUBO-based subset selection, enhancing weak supervision in Arabic framing detection.

Findings

Selected data subsets are more reliable and less redundant.

The approach improves transferability of sentiment models.

No degradation observed in baseline performance.

Abstract

Framing detection in Arabic social media is difficult due to interpretive ambiguity, cultural grounding, and limited reliable supervision. Existing LLM-based weak supervision methods typically rely on label aggregation, which is brittle when annotations are few and socially dependent. We propose a reliability-aware weak supervision framework that shifts the focus from label fusion to data curation. A small multi-agent LLM pipeline, two framers, a critic, and a discriminator, treats disagreement and reasoning quality as epistemic signals and produces instance-level reliability estimates. These estimates guide a QUBO-based subset selection procedure that enforces frame balance while reducing redundancy. Intrinsic diagnostics and an out-of-domain Arabic sentiment transfer test show that the selected subsets are more reliable and encode non-random, transferable structure, without degrading strong text-only baselines.