Attribute-Grounded Selective Reasoning for Artwork Emotion Understanding with Multimodal Large Language Models

TL;DR

This paper introduces Attribute-Grounded Selective Reasoning (AGSR) for artwork emotion understanding, leveraging a new annotated dataset and a supervised multi-agent framework to improve interpretability and accuracy of emotional analysis.

Contribution

It proposes a novel AGSR framework with formal-attribute bottleneck-guided reasoning and extends EmoArt with salience annotations for better attribute selection.

Findings

FAB-G improves emotion, arousal, and valence prediction accuracy.

FAB-G produces more compact and human-aligned explanations.

Attribute-grounded salience transferability across datasets is demonstrated.

Abstract

Multimodal large language models (MLLMs) can produce fluent artwork emotion explanations, but they often suffer from attribute flooding: they enumerate many visible formal attributes without identifying which cues actually support the affective judgment. We therefore formulate artwork emotion understanding as Attribute-Grounded Selective Reasoning (AGSR), where predefined formal attributes serve as evidence units and only emotionally operative attributes should enter the final interpretation. To make this problem measurable, we extend EmoArt, originally introduced at ACM MM 2025 as a 132,664-artwork resource with content, formal-attribute, valence-arousal, and emotion annotations, by adding a 1,400-artwork human salience extension annotated by 15 art-trained annotators. This extension provides instance-level supervision for distinguishing attributes that are merely present from those that are emotionally salient. We further propose FAB-G (Formal-Attribute Bottleneck-Guided reasoning), a supervised multi-agent framework that first predicts attribute-level salience and then constrains downstream emotional analysis to the retained cues. Experiments show that FAB-G yields consistent gains in emotion, arousal, and valence prediction, achieves stronger agreement with human-marked salient attributes under Dice and Tversky metrics, and produces substantially more compact final explanations than prompting-based baselines. Cross-dataset evaluation further suggests that attribute-grounded salience selection transfers beyond the source distribution of EmoArt, while also revealing attribute-specific boundary cases. The dataset and project page are available at https://zhiliangzhang.github.io/EmoArt-130k/