Song Aesthetics Evaluation with Multi-Stem Attention and Hierarchical Uncertainty Modeling

TL;DR

This paper introduces a novel framework for automated song aesthetics evaluation using multi-stem attention and hierarchical uncertainty modeling, effectively capturing musical nuances and human perception complexities.

Contribution

It proposes two innovative modules, MSAF and HiGIA, to better model musical features and score uncertainties, advancing beyond traditional MOS prediction methods.

Findings

Outperforms existing models on SongEval and internal datasets

Effectively captures complex musical features and human perception nuances

Achieves stronger performance in multi-dimensional song aesthetics evaluation

Abstract

Music generative artificial intelligence (AI) is rapidly expanding music content, necessitating automated song aesthetics evaluation. However, existing studies largely focus on speech, audio or singing quality, leaving song aesthetics underexplored. Moreover, conventional approaches often predict a precise Mean Opinion Score (MOS) value directly, which struggles to capture the nuances of human perception in song aesthetics evaluation. This paper proposes a song-oriented aesthetics evaluation framework, featuring two novel modules: 1) Multi-Stem Attention Fusion (MSAF) builds bidirectional cross-attention between mixture-vocal and mixture-accompaniment pairs, fusing them to capture complex musical features; 2) Hierarchical Granularity-Aware Interval Aggregation (HiGIA) learns multi-granularity score probability distributions, aggregates them into a score interval, and applies a regression within the interval to produce the final score. We evaluated on two datasets of full-length songs: SongEval dataset (AI-generated) and an internal aesthetics dataset (human-created), and compared with two state-of-the-art (SOTA) models. Results show that the proposed method achieves stronger performance for multi-dimensional song aesthetics evaluation.