I know why you like this movie: Interpretable Efficient Multimodal Recommender

TL;DR

This paper demonstrates that the EMDE recommender model, despite its compressed input representation, can be interpreted in a white-box manner, revealing how different modalities influence movie recommendations.

Contribution

It introduces a method to interpret EMDE's recommendations by leveraging its item retrieval properties, enabling understanding of multimodal influences.

Findings

Interpretability of EMDE in a white-box setting is feasible.

Multimodal features like text, images, and categorical data significantly influence recommendations.

The approach enhances transparency in complex recommender systems.

Abstract

Recently, the Efficient Manifold Density Estimator (EMDE) model has been introduced. The model exploits Local Sensitive Hashing and Count-Min Sketch algorithms, combining them with a neural network to achieve state-of-the-art results on multiple recommender datasets. However, this model ingests a compressed joint representation of all input items for each user/session, so calculating attributions for separate items via gradient-based methods seems not applicable. We prove that interpreting this model in a white-box setting is possible thanks to the properties of EMDE item retrieval method. By exploiting multimodal flexibility of this model, we obtain meaningful results showing the influence of multiple modalities: text, categorical features, and images, on movie recommendation output.