FITMM: Adaptive Frequency-Aware Multimodal Recommendation via Information-Theoretic Representation Learning

TL;DR

This paper introduces FITMM, a spectral information-theoretic framework for multimodal recommendation that leverages frequency decomposition to improve user preference modeling and recommendation accuracy.

Contribution

It proposes a novel frequency-aware spectral decomposition approach using an information bottleneck, enabling better fusion and regularization of multimodal data in recommendation systems.

Findings

FITMM outperforms state-of-the-art baselines on three real-world datasets.

The frequency-domain IB regularization improves model generalization.

Spectral decomposition enhances multimodal feature alignment and representation.

Abstract

Multimodal recommendation aims to enhance user preference modeling by leveraging rich item content such as images and text. Yet dominant systems fuse modalities in the spatial domain, obscuring the frequency structure of signals and amplifying misalignment and redundancy. We adopt a spectral information-theoretic view and show that, under an orthogonal transform that approximately block-diagonalizes bandwise covariances, the Gaussian Information Bottleneck objective decouples across frequency bands, providing a principled basis for separate-then-fuse paradigm. Building on this foundation, we propose FITMM, a Frequency-aware Information-Theoretic framework for multimodal recommendation. FITMM constructs graph-enhanced item representations, performs modality-wise spectral decomposition to obtain orthogonal bands, and forms lightweight within-band multimodal components. A residual, task-adaptive gate aggregates bands into the final representation. To control redundancy and improve generalization, we regularize training with a frequency-domain IB term that allocates capacity across bands (Wiener-like shrinkage with shut-off of weak bands). We further introduce a cross-modal spectral consistency loss that aligns modalities within each band. The model is jointly optimized with the standard recommendation loss. Extensive experiments on three real-world datasets demonstrate that FITMM consistently and significantly outperforms advanced baselines.