Analyzing Images for Music Recommendation

TL;DR

This paper presents a deep learning-based method for analyzing images to improve music recommendation, differentiating between artwork and photographs, and validating the approach through subjective assessments.

Contribution

It introduces a novel image analysis approach that classifies artworks and photographs differently for music recommendation, leveraging deep models to learn perceptually relevant features.

Findings

Deep models effectively classify artwork and photographs.

Subjective assessments confirm improved music recommendation quality.

Perceptually relevant features are learned by the models.

Abstract

Experiencing images with suitable music can greatly enrich the overall user experience. The proposed image analysis method treats an artwork image differently from a photograph image. Automatic image classification is performed using deep-learning based models. An illustrative analysis showcasing the ability of our deep-models to inherently learn and utilize perceptually relevant features when classifying artworks is also presented. The Mean Opinion Score (MOS) obtained from subjective assessments of the respective image and recommended music pairs supports the effectiveness of our approach.