MM-SCALE: Grounded Multimodal Moral Reasoning via Scalar Judgment and Listwise Alignment

TL;DR

This paper introduces MM-SCALE, a large-scale dataset that uses scalar ratings and explicit grounding to improve multimodal moral reasoning in vision-language models, enabling finer alignment with human moral judgments.

Contribution

We propose MM-SCALE, a novel dataset with scalar moral ratings and grounding, facilitating better alignment and calibration of VLMs for moral reasoning tasks.

Findings

VLMs fine-tuned on MM-SCALE show higher ranking fidelity.

Models achieve more stable safety calibration.

Scalar supervision provides richer alignment signals.

Abstract

Vision-Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts. Prior works typically rely on binary or pairwise supervision, which often fail to capture the continuous and pluralistic nature of human moral reasoning. We present MM-SCALE (Multimodal Moral Scale), a large-scale dataset for aligning VLMs with human moral preferences through 5-point scalar ratings and explicit modality grounding. Each image-scenario pair is annotated with moral acceptability scores and grounded reasoning labels by humans using an interface we tailored for data collection, enabling listwise preference optimization over ranked scenario sets. By moving from discrete to scalar supervision, our framework provides richer alignment signals and finer calibration of multimodal moral reasoning. Experiments show that VLMs fine-tuned on MM-SCALE achieve higher ranking fidelity and more stable safety calibration than those trained with binary signals.