It is Never Too Late to Mend: Separate Learning for Multimedia Recommendation

TL;DR

This paper introduces Separate Learning (SEA), a novel framework for multimedia recommendation that effectively learns modal-unique and modal-generic features by leveraging mutual information techniques, outperforming previous alignment-based methods.

Contribution

The paper proposes a new Separate Learning framework that addresses limitations of existing methods by explicitly learning modal-unique and modal-generic features using mutual information, with extensive experimental validation.

Findings

SEA outperforms existing methods on three datasets.

Mutual information-based learning improves feature quality.

The framework demonstrates strong generalization capabilities.

Abstract

Multimedia recommendation, which incorporates various modalities (e.g., images, texts, etc.) into user or item representation to improve recommendation quality, and self-supervised learning carries multimedia recommendation to a plateau of performance, because of its superior performance in aligning different modalities. However, more and more research finds that aligning all modal representations is suboptimal because it damages the unique attributes of each modal. These studies use subtraction and orthogonal constraints in geometric space to learn unique parts. However, our rigorous analysis reveals the flaws in this approach, such as that subtraction does not necessarily yield the desired modal-unique and that orthogonal constraints are ineffective in user and item high-dimensional representation spaces. To make up for the previous weaknesses, we propose Separate Learning (SEA) for multimedia recommendation, which mainly includes mutual information view of modal-unique and -generic learning. Specifically, we first use GNN to learn the representations of users and items in different modalities and split each modal representation into generic and unique parts. We employ contrastive log-ratio upper bound to minimize the mutual information between the general and unique parts within the same modality, to distance their representations, thus learning modal-unique features. Then, we design Solosimloss to maximize the lower bound of mutual information, to align the general parts of different modalities, thus learning more high-quality modal-generic features. Finally, extensive experiments on three datasets demonstrate the effectiveness and generalization of our proposed framework. The code is available at SEA and the full training record of the main experiment.