Survey on the Evaluation of Generative Models in Music

TL;DR

This paper provides a comprehensive review of methods and metrics used to evaluate generative music models, highlighting interdisciplinary perspectives and discussing their benefits and limitations.

Contribution

It offers an extensive survey of evaluation techniques for generative music models, integrating insights from musicology, engineering, and HCI.

Findings

Summarizes evaluation targets and methodologies in generative music

Analyzes benefits and limitations of different evaluation approaches

Bridges perspectives across disciplines in music AI evaluation

Abstract

Research on generative systems in music has seen considerable attention and growth in recent years. A variety of attempts have been made to systematically evaluate such systems. We present an interdisciplinary review of the common evaluation targets, methodologies, and metrics for the evaluation of both system output and model use, covering subjective and objective approaches, qualitative and quantitative approaches, as well as empirical and computational methods. We examine the benefits and limitations of these approaches from a musicological, an engineering, and an HCI perspective.