SongSong: A Time Phonograph for Chinese SongCi Music from Thousand of Years Away

TL;DR

SongSong is a novel music generation model that restores ancient Chinese SongCi, combining melody prediction, voice and accompaniment generation, and dataset creation, achieving high-quality results for this traditional genre.

Contribution

We introduce SongSong, the first model capable of generating authentic Chinese SongCi music, along with the OpenSongSong dataset for training and evaluation.

Findings

SongSong outperforms existing platforms in quality and authenticity.

The OpenSongSong dataset contains 29.9 hours of ancient Chinese SongCi music.

Subjective and objective evaluations confirm SongSong's superior performance.

Abstract

Recently, there have been significant advancements in music generation. However, existing models primarily focus on creating modern pop songs, making it challenging to produce ancient music with distinct rhythms and styles, such as ancient Chinese SongCi. In this paper, we introduce SongSong, the first music generation model capable of restoring Chinese SongCi to our knowledge. Our model first predicts the melody from the input SongCi, then separately generates the singing voice and accompaniment based on that melody, and finally combines all elements to create the final piece of music. Additionally, to address the lack of ancient music datasets, we create OpenSongSong, a comprehensive dataset of ancient Chinese SongCi music, featuring 29.9 hours of compositions by various renowned SongCi music masters. To assess SongSong's proficiency in performing SongCi, we randomly select 85 SongCi sentences that were not part of the training set for evaluation against SongSong and music generation platforms such as Suno and SkyMusic. The subjective and objective outcomes indicate that our proposed model achieves leading performance in generating high-quality SongCi music.