ScripTONES: Sentiment-Conditioned Music Generation for Movie Scripts

TL;DR

This paper introduces ScripTONES, a two-stage system that analyzes movie script sentiment and generates matching piano music, aiming to automate film scoring for small-scale creators.

Contribution

It presents a novel pipeline combining sentiment analysis with conditional music generation, and explores improvements in sentiment-conditioning for VAE architectures.

Findings

Sentiment analysis effectively encodes scene emotion in valence-arousal space.

Conditional music generation can produce piano MIDI matching scene sentiment.

Proposed methods improve sentiment-conditioning in VAE-based music models.

Abstract

Film scores are considered an essential part of the film cinematic experience, but the process of film score generation is often expensive and infeasible for small-scale creators. Automating the process of film score composition would provide useful starting points for music in small projects. In this paper, we propose a two-stage pipeline for generating music from a movie script. The first phase is the Sentiment Analysis phase where the sentiment of a scene from the film script is encoded into the valence-arousal continuous space. The second phase is the Conditional Music Generation phase which takes as input the valence-arousal vector and conditionally generates piano MIDI music to match the sentiment. We study the efficacy of various music generation architectures by performing a qualitative user survey and propose methods to improve sentiment-conditioning in VAE architectures.