MuSpike: A Benchmark and Evaluation Framework for Symbolic Music Generation with Spiking Neural Networks

TL;DR

MuSpike introduces a comprehensive benchmark and evaluation framework for assessing spiking neural network models in symbolic music generation, combining objective metrics with human perceptual studies to address existing gaps.

Contribution

This work presents the first systematic benchmark and evaluation framework for SNN-based symbolic music generation, including new subjective metrics and analysis of model and listener diversity.

Findings

Different SNN models show distinct strengths across evaluation metrics.

Musicians and non-musicians perceive AI-generated music differently.

Objective metrics do not always align with human perceptual judgments.

Abstract

Symbolic music generation has seen rapid progress with artificial neural networks, yet remains underexplored in the biologically plausible domain of spiking neural networks (SNNs), where both standardized benchmarks and comprehensive evaluation methods are lacking. To address this gap, we introduce MuSpike, a unified benchmark and evaluation framework that systematically assesses five representative SNN architectures (SNN-CNN, SNN-RNN, SNN-LSTM, SNN-GAN and SNN-Transformer) across five typical datasets, covering tonal, structural, emotional, and stylistic variations. MuSpike emphasizes comprehensive evaluation, combining established objective metrics with a large-scale listening study. We propose new subjective metrics, targeting musical impression, autobiographical association, and personal preference, that capture perceptual dimensions often overlooked in prior work. Results reveal that (1) different SNN models exhibit distinct strengths across evaluation dimensions; (2) participants with different musical backgrounds exhibit diverse perceptual patterns, with experts showing greater tolerance toward AI-composed music; and (3) a noticeable misalignment exists between objective and subjective evaluations, highlighting the limitations of purely statistical metrics and underscoring the value of human perceptual judgment in assessing musical quality. MuSpike provides the first systematic benchmark and systemic evaluation framework for SNN models in symbolic music generation, establishing a solid foundation for future research into biologically plausible and cognitively grounded music generation.