SurpriseNet: Melody Harmonization Conditioning on User-controlled Surprise Contours

TL;DR

SurpriseNet introduces a user-controllable melody harmonization framework that leverages surprise contours derived from entropy variations, enabling the generation of harmonies aligned with specified surprise levels.

Contribution

It proposes a novel CVAE-based model that incorporates surprise contours for melody harmonization, allowing explicit control over harmonic surprise levels.

Findings

The model effectively matches surprise contours with high correlation.

Performance is comparable to state-of-the-art melody harmonization models.

The framework enables flexible, user-controlled harmonic generation.

Abstract

The surprisingness of a song is an essential and seemingly subjective factor in determining whether the listener likes it. With the help of information theory, it can be described as the transition probability of a music sequence modeled as a Markov chain. In this study, we introduce the concept of deriving entropy variations over time, so that the surprise contour of each chord sequence can be extracted. Based on this, we propose a user-controllable framework that uses a conditional variational autoencoder (CVAE) to harmonize the melody based on the given chord surprise indication. Through explicit conditions, the model can randomly generate various and harmonic chord progressions for a melody, and the Spearman's correlation and p-value significance show that the resulting chord progressions match the given surprise contour quite well. The vanilla CVAE model was evaluated in a basic melody harmonization task (no surprise control) in terms of six objective metrics. The results of experiments on the Hooktheory Lead Sheet Dataset show that our model achieves performance comparable to the state-of-the-art melody harmonization model.