Enhancing Multimodal Recommendations with Vision-Language Models and Information-Aware Fusion

TL;DR

VIRAL introduces a novel multimodal recommendation framework that leverages fine-grained visual descriptions and an information-aware fusion strategy to improve item representations and recommendation accuracy.

Contribution

The paper presents VIRAL, a new framework combining vision-language models and information theory for better multimodal fusion in recommendations.

Findings

VIRAL outperforms existing multimodal recommendation baselines.

Visual feature contribution is significantly enhanced.

The framework effectively disentangles shared and unique modality information.

Abstract

Recent advances in multimodal recommendation (MMR) highlight the potential of integrating visual and textual content to enrich item representations. However, existing methods often rely on coarse visual features and naive fusion strategies, resulting in redundant or misaligned representations. From an information-theoretic perspective, effective fusion should balance unique, shared, and redundant modality information to preserve complementary cues. To this end, we propose VIRAL, a novel Vision-Language and Information-aware Recommendation framework that enhances multimodal fusion through two components: (i) a VLM-based visual enrichment module that generates fine-grained, title-guided descriptions for semantically aligned image representations, and (ii) an information-aware fusion module inspired by Partial Information Decomposition (PID) to disentangle and integrate complementary signals. Experiments on three Amazon datasets show that VIRAL consistently outperforms strong multimodal baselines and substantially improves the contribution of visual features.