VOGUE: A Multimodal Dataset for Conversational Recommendation in Fashion

TL;DR

VOGUE is a comprehensive multimodal dataset for fashion conversational recommendation, enabling detailed evaluation of dialogue inference, preference alignment, and user satisfaction, highlighting current model limitations.

Contribution

Introduces VOGUE, a detailed dataset with real human dialogues, visual data, and ratings, addressing limitations of existing resources for multimodal conversational recommendation research.

Findings

Reveals distinctive visually grounded dialogue dynamics.

Shows multimodal models approach human alignment but struggle with preference generalization.

Identifies systematic rating distribution errors in current models.

Abstract

Multimodal conversational recommendation has emerged as a promising paradigm for delivering personalized experiences through natural dialogue enriched by visual and contextual grounding. Yet, current multimodal conversational recommendation datasets remain limited: existing resources either simulate conversations, omit user history, or fail to collect sufficiently detailed feedback, all of which constrain the types of research and evaluation they support. To address these gaps, we introduce VOGUE, a novel dataset of 60 humanhuman dialogues in realistic fashion shopping scenarios. Each dialogue is paired with a shared visual catalogue, item metadata, user fashion profiles and histories, and post-conversation ratings from both Seekers and Assistants. This design enables rigorous evaluation of conversational inference, including not only alignment between predicted and ground-truth preferences, but also calibration against full rating distributions and comparison with explicit and implicit user satisfaction signals. Our initial analyses of VOGUE reveal distinctive dynamics of visually grounded dialogue. For example, recommenders frequently suggest items simultaneously in feature-based groups, which creates distinct conversational phases bridged by Seeker critiques and refinements. Benchmarking multimodal large language models against human recommenders shows that while MLLMs approach human-level alignment in aggregate, they exhibit systematic distribution errors in reproducing human ratings and struggle to generalize preference inference beyond explicitly discussed items. These findings establish VOGUE as both a unique resource for studying multimodal conversational systems and as a challenge dataset beyond the current recommendation capabilities of existing top-tier multimodal foundation models such as GPT-4o-mini, GPT-5-mini, and Gemini-2.5-Flash.