Multi-objective Hyper-parameter Optimization of Behavioral Song Embeddings

TL;DR

This paper explores multi-objective hyper-parameter optimization for behavioral song embeddings using Word2Vec, improving recommendation quality and balancing performance across song popularity levels.

Contribution

It introduces new optimization objectives and metrics, and demonstrates the benefits of multi-objective optimization over single-objective approaches in music embedding tasks.

Findings

Multi-objective optimization mitigates side effects on non-optimized metrics.

Embedding quality is anti-correlated with song popularity.

Optimization improves downstream play prediction performance.

Abstract

Song embeddings are a key component of most music recommendation engines. In this work, we study the hyper-parameter optimization of behavioral song embeddings based on Word2Vec on a selection of downstream tasks, namely next-song recommendation, false neighbor rejection, and artist and genre clustering. We present new optimization objectives and metrics to monitor the effects of hyper-parameter optimization. We show that single-objective optimization can cause side effects on the non optimized metrics and propose a simple multi-objective optimization to mitigate these effects. We find that next-song recommendation quality of Word2Vec is anti-correlated with song popularity, and we show how song embedding optimization can balance performance across different popularity levels. We then show potential positive downstream effects on the task of play prediction. Finally, we provide useful insights on the effects of training dataset scale by testing hyper-parameter optimization on an industry-scale dataset.