REMAST: Real-time Emotion-based Music Arrangement with Soft Transition
Zihao Wang, Le Ma, Chen Zhang, Bo Han, Yunfei Xu, Yikai Wang, Xinyi, Chen, HaoRong Hong, Wenbo Liu, Xinda Wu, Kejun Zhang

TL;DR
REMAST is a real-time music arrangement system that dynamically adapts to changing emotions, ensuring both emotional fit and smooth transitions by fusing past and current emotional cues and leveraging domain knowledge features.
Contribution
The paper introduces REMAST, a novel system that balances emotion real-time fit with smooth transitions in music generation using emotion fusion, feature enhancement, and semi-supervised learning.
Findings
REMAST outperforms existing methods in objective metrics.
REMAST achieves superior subjective evaluation scores.
The system maintains emotional coherence with real-time adaptation.
Abstract
Music as an emotional intervention medium has important applications in scenarios such as music therapy, games, and movies. However, music needs real-time arrangement according to changing emotions, bringing challenges to balance emotion real-time fit and soft emotion transition due to the fine-grained and mutable nature of the target emotion. Existing studies mainly focus on achieving emotion real-time fit, while the issue of smooth transition remains understudied, affecting the overall emotional coherence of the music. In this paper, we propose REMAST to address this trade-off. Specifically, we recognize the last timestep's music emotion and fuse it with the current timestep's input emotion. The fused emotion then guides REMAST to generate the music based on the input melody. To adjust music similarity and emotion real-time fit flexibly, we downsample the original melody and feed it…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Neuroscience and Music Perception · Music Technology and Sound Studies
