Formalizing Multimedia Recommendation through Multimodal Deep Learning

TL;DR

This paper formalizes a universal multimodal schema for multimedia recommendation, reviews recent approaches, and benchmarks algorithms to guide future multimodal recommender system development.

Contribution

It introduces a comprehensive multimodal schema, reviews eight years of literature, and provides benchmarking guidelines for multimedia recommendation systems.

Findings

Proposes a general multimodal schema for multimedia recommendation

Provides a benchmarking analysis of recent algorithms

Offers guidelines for designing next-generation multimodal recommenders

Abstract

Recommender systems (RSs) offer personalized navigation experiences on online platforms, but recommendation remains a challenging task, particularly in specific scenarios and domains. Multimodality can help tap into richer information sources and construct more refined user/item profiles for recommendations. However, existing literature lacks a shared and universal schema for modeling and solving the recommendation problem through the lens of multimodality. This work aims to formalize a general multimodal schema for multimedia recommendation. It provides a comprehensive literature review of multimodal approaches for multimedia recommendation from the last eight years, outlines the theoretical foundations of a multimodal pipeline, and demonstrates its rationale by applying it to selected state-of-the-art approaches. The work also conducts a benchmarking analysis of recent algorithms for multimedia recommendation within Elliot, a rigorous framework for evaluating recommender systems. The main aim is to provide guidelines for designing and implementing the next generation of multimodal approaches in multimedia recommendation.