Train Once, Deploy Anywhere: Matryoshka Representation Learning for Multimodal Recommendation

TL;DR

This paper introduces fMRLRec, a lightweight, scalable multimodal recommendation framework that captures item features at multiple granularities, enabling efficient, one-time training for adaptable recommendation models across diverse datasets.

Contribution

The paper presents a novel Matryoshka representation learning framework that efficiently integrates multimodal features at different granularities for scalable recommendation.

Findings

fMRLRec outperforms state-of-the-art methods on benchmark datasets.

It reduces memory requirements through linear transformations.

The framework scales to various dimensions with one-time training.

Abstract

Despite recent advancements in language and vision modeling, integrating rich multimodal knowledge into recommender systems continues to pose significant challenges. This is primarily due to the need for efficient recommendation, which requires adaptive and interactive responses. In this study, we focus on sequential recommendation and introduce a lightweight framework called full-scale Matryoshka representation learning for multimodal recommendation (fMRLRec). Our fMRLRec captures item features at different granularities, learning informative representations for efficient recommendation across multiple dimensions. To integrate item features from diverse modalities, fMRLRec employs a simple mapping to project multimodal item features into an aligned feature space. Additionally, we design an efficient linear transformation that embeds smaller features into larger ones, substantially reducing memory requirements for large-scale training on recommendation data. Combined with improved state space modeling techniques, fMRLRec scales to different dimensions and only requires one-time training to produce multiple models tailored to various granularities. We demonstrate the effectiveness and efficiency of fMRLRec on multiple benchmark datasets, which consistently achieves superior performance over state-of-the-art baseline methods. We make our code and data publicly available at https://github.com/yueqirex/fMRLRec.