Multi-modal Food Recommendation using Clustering and Self-supervised Learning

TL;DR

This paper introduces CLUSSL, a novel multi-modal food recommendation framework that uses clustering and self-supervised learning to improve recommendation accuracy by capturing semantic relations between recipes.

Contribution

The paper proposes CLUSSL, which employs modality-specific graphs and self-supervised learning to better model semantic relations in multi-modal food recommendation systems.

Findings

CLUSSL outperforms state-of-the-art benchmarks in experiments.

Graph-based representations enhance semantic understanding of recipes.

Self-supervised learning fosters independence between modality-specific features.

Abstract

Food recommendation systems serve as pivotal components in the realm of digital lifestyle services, designed to assist users in discovering recipes and food items that resonate with their unique dietary predilections. Typically, multi-modal descriptions offer an exhaustive profile for each recipe, thereby ensuring recommendations that are both personalized and accurate. Our preliminary investigation of two datasets indicates that pre-trained multi-modal dense representations might precipitate a deterioration in performance compared to ID features when encapsulating interactive relationships. This observation implies that ID features possess a relative superiority in modeling interactive collaborative signals. Consequently, contemporary cutting-edge methodologies augment ID features with multi-modal information as supplementary features, overlooking the latent semantic relations between recipes. To rectify this, we present CLUSSL, a novel food recommendation framework that employs clustering and self-supervised learning. Specifically, CLUSSL formulates a modality-specific graph tailored to each modality with discrete/continuous features, thereby transforming semantic features into structural representation. Furthermore, CLUSSL procures recipe representations pertinent to different modalities via graph convolutional operations. A self-supervised learning objective is proposed to foster independence between recipe representations derived from different unimodal graphs. Comprehensive experiments on real-world datasets substantiate that CLUSSL consistently surpasses state-of-the-art recommendation benchmarks in performance.