Revisiting Content-Based Music Recommendation: Efficient Feature Aggregation from Large-Scale Music Models

TL;DR

This paper introduces TASTE, a multimodal music recommendation dataset and framework, demonstrating the effectiveness of large-scale audio encoders and a new feature aggregation method for improved recommendation performance.

Contribution

It presents a new multimodal dataset, a benchmarking framework, and the MuQ-token method for efficient multi-layer audio feature integration in music recommendation.

Findings

Audio representations significantly improve recommendation tasks.

MuQ-token outperforms other feature integration methods.

Content-based approaches are validated as effective for music recommendation.

Abstract

Music Recommendation Systems (MRSs) are a cornerstone of modern streaming platforms. Existing recommendation models, spanning both recall and ranking stages, predominantly rely on collaborative filtering, which fails to exploit the intrinsic characteristics of audio and consequently leads to suboptimal performance, particularly in cold-start scenarios. However, existing music recommendation datasets often lack rich multimodal information, such as raw audio signals and descriptive textual metadata. Moreover, current recommender system evaluation frameworks remain inadequate, as they neither fully leverage multimodal information nor support a diverse range of algorithms, especially multimodal methods. To address these limitations, we propose TASTE, a comprehensive dataset and benchmarking framework designed to highlight the role of multimodal information in music recommendation. Our dataset integrates both audio and textual modalities. By leveraging recent large-scale self-supervised music encoders, we demonstrate the substantial value of the extracted audio representations across recommendation tasks, including candidate recall and CTR. In addition, we introduce the \textbf{MuQ-token} method, which enables more efficient integration of multi-layer audio features. This method consistently outperforms other feature integration techniques across various settings. Overall, our results not only validate the effectiveness of content-driven approaches but also provide a highly effective and reusable multimodal foundation for future research. Code is available at https://github.com/zreach/TASTE