Just Ask for Music (JAM): Multimodal and Personalized Natural Language Music Recommendation

TL;DR

JAM is a lightweight, multimodal, and personalized natural language music recommendation framework that models user preferences as vector translations, effectively capturing complex queries and long-term user interests with high accuracy.

Contribution

This paper introduces JAM, a novel, scalable framework that models user-query-item interactions as vector translations, integrating multimodal features and long-term preferences for music recommendation.

Findings

JAM achieves accurate recommendations in experiments.

JAM produces intuitive, practical representations.

JAM can be integrated into existing systems.

Abstract

Natural language interfaces offer a compelling approach for music recommendation, enabling users to express complex preferences conversationally. While Large Language Models (LLMs) show promise in this direction, their scalability in recommender systems is limited by high costs and latency. Retrieval-based approaches using smaller language models mitigate these issues but often rely on single-modal item representations, overlook long-term user preferences, and require full model retraining, posing challenges for real-world deployment. In this paper, we present JAM (Just Ask for Music), a lightweight and intuitive framework for natural language music recommendation. JAM models user-query-item interactions as vector translations in a shared latent space, inspired by knowledge graph embedding methods like TransE. To capture the complexity of music and user intent, JAM aggregates multimodal item features via cross-attention and sparse mixture-of-experts. We also introduce JAMSessions, a new dataset of over 100k user-query-item triples with anonymized user/item embeddings, uniquely combining conversational queries and user long-term preferences. Our results show that JAM provides accurate recommendations, produces intuitive representations suitable for practical use cases, and can be easily integrated with existing music recommendation stacks.