Multimodal Difference Learning for Sequential Recommendation

TL;DR

This paper introduces MDSRec, a novel framework that models multimodal differences in sequential recommendation by constructing modal-aware relation graphs and using interest-centered attention, improving recommendation accuracy.

Contribution

The paper proposes a new multimodal difference learning approach that explicitly models differences across modalities for sequential recommendation tasks.

Findings

MDSRec outperforms state-of-the-art baselines on five real-world datasets.

Constructing modal-aware relation graphs enhances item representation.

Interest-centered attention effectively captures user interest variations across modalities.

Abstract

Sequential recommendations have drawn significant attention in modeling the user's historical behaviors to predict the next item. With the booming development of multimodal data (e.g., image, text) on internet platforms, sequential recommendation also benefits from the incorporation of multimodal data. Most methods introduce modal features of items as side information and simply concatenates them to learn unified user interests. Nevertheless, these methods encounter the limitation in modeling multimodal differences. We argue that user interests and item relationships vary across different modalities. To address this problem, we propose a novel Multimodal Difference Learning framework for Sequential Recommendation, MDSRec for brevity. Specifically, we first explore the differences in item relationships by constructing modal-aware item relation graphs with behavior signal to enhance item representations. Then, to capture the differences in user interests across modalities, we design a interest-centralized attention mechanism to independently model user sequence representations in different modalities. Finally, we fuse the user embeddings from multiple modalities to achieve accurate item recommendation. Experimental results on five real-world datasets demonstrate the superiority of MDSRec over state-of-the-art baselines and the efficacy of multimodal difference learning.