GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysis

TL;DR

This paper introduces GRCF, a novel two-stage framework for multimodal sentiment analysis that improves ordinal ranking and calibration, leading to state-of-the-art results and better handling of difficult samples.

Contribution

The paper proposes a two-stage ranking and calibration framework that adaptively focuses on hard samples and reflects semantic distances, advancing multimodal sentiment analysis beyond point-wise regression.

Findings

Achieves state-of-the-art performance on regression benchmarks.

Effectively generalizes to classification tasks like humor and sarcasm detection.

Improves stability and correlation alignment in sentiment predictions.

Abstract

Most Multimodal Sentiment Analysis research has focused on point-wise regression. While straightforward, this approach is sensitive to label noise and neglects whether one sample is more positive than another, resulting in unstable predictions and poor correlation alignment. Pairwise ordinal learning frameworks emerged to address this gap, capturing relative order by learning from comparisons. Yet, they introduce two new trade-offs: First, they assign uniform importance to all comparisons, failing to adaptively focus on hard-to-rank samples. Second, they employ static ranking margins, which fail to reflect the varying semantic distances between sentiment groups. To address this, we propose a Two-Stage Group-wise Ranking and Calibration Framework (GRCF) that adapts the philosophy of Group Relative Policy Optimization (GRPO). Our framework resolves these trade-offs by simultaneously preserving relative ordinal structure, ensuring absolute score calibration, and adaptively focusing on difficult samples. Specifically, Stage 1 introduces a GRPO-inspired Advantage-Weighted Dynamic Margin Ranking Loss to build a fine-grained ordinal structure. Stage 2 then employs an MAE-driven objective to align prediction magnitudes. To validate its generalizability, we extend GRCF to classification tasks, including multimodal humor detection and sarcasm detection. GRCF achieves state-of-the-art performance on core regression benchmarks, while also showing strong generalizability in classification tasks.