Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression

TL;DR

This paper introduces a distribution-aware reinforcement learning approach for multimodal large language models to improve their performance on long-tailed numerical regression tasks, addressing the limitations of traditional fine-tuning methods.

Contribution

It proposes a novel reinforcement learning framework that incorporates batch-level distributional supervision, significantly enhancing tail performance without changing model architecture.

Findings

Consistent improvements over existing methods on long-tailed regression benchmarks.

Strong gains observed in medium- and few-shot regimes.

The approach effectively aligns predicted and ground-truth distributions.

Abstract

Multimodal large language models (MLLMs) struggle with numerical regression under long-tailed target distributions. Token-level supervised fine-tuning (SFT) and point-wise regression rewards bias learning toward high-density regions, leading to regression-to-the-mean behavior and poor tail performance. We identify the lack of cross-sample relational supervision as a key limitation of existing MLLM training paradigms. To address it, we propose a distribution-aware reinforcement learning framework based on Group Relative Policy Optimization, which introduces batch-level comparison-based supervision via the Concordance Correlation Coefficient-based reward to align predicted and ground-truth distributions in terms of correlation, scale, and mean. The framework is plug-and-play, requiring no architectural modification. Experiments on a unified suite of long-tailed regression benchmarks show consistent improvements over SFT and existing MLLM regression methods, with particularly strong gains in medium- and few-shot regimes.