Towards Trustworthy Multimodal Recommendation

TL;DR

This paper introduces a plug-and-play modality rectification method to improve trustworthiness and robustness of multimodal recommendation systems against unreliable content and noisy signals, validated through extensive experiments.

Contribution

It proposes a novel soft matching rectification component for mitigating untrustworthy modality features in multimodal recommenders, without requiring architectural changes.

Findings

Rectification improves robustness against modality corruption.

Pseudo interactions can either help or hurt performance depending on context.

Propagation-graph pseudo edges influence robustness through message passing.

Abstract

Recent advances in multimodal recommendation have demonstrated the effectiveness of incorporating visual and textual content into collaborative filtering. However, real-world deployments raise an increasingly important yet underexplored issue: trustworthiness. On modern e-commerce platforms, multimodal content can be misleading or unreliable (e.g., visually inconsistent product images or click-bait titles), injecting untrustworthy signals into multimodal representations and making existing recommenders brittle under modality corruption. In this work, we take a step towards trustworthy multimodal recommendation from both a method and an analysis perspective. First, we propose a plug-and-play modality-level rectification component that mitigates untrustworthy modality features by learning soft correspondences between items and multimodal features. Using lightweight projections and Sinkhorn-based soft matching, the rectification suppresses mismatched modality signals while preserving semantic consistency, and can be integrated into existing multimodal recommenders without architectural modifications. Second, we present two practical insights on interaction-level trustworthiness under noisy collaborative signals: (i) training-set pseudo interactions can help or hurt performance under noise depending on prior-signal alignment; and (ii) propagation-graph pseudo edges can also help or hurt robustness, as message passing may amplify misalignment. Extensive experiments on multiple datasets and backbones under varying corruption levels demonstrate improved robustness from modality rectification and validate the above interaction-level observations.