BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Features

TL;DR

BandCondiNet is a parallel Transformer-based model for conditional multitrack music generation that improves fidelity, structure, and inter-track harmony using multi-view features and specialized modules, outperforming existing models.

Contribution

The paper introduces BandCondiNet, a novel parallel Transformer architecture with multi-view features and modules for enhanced multitrack music generation, addressing fidelity and harmony challenges.

Findings

Outperforms other models on multiple metrics in fidelity and speed.

Achieves superior subjective quality on longer datasets.

Effectively models musical structure and inter-track harmony.

Abstract

Conditional music generation offers significant advantages in terms of user convenience and control, presenting great potential in AI-generated content research. However, building conditional generative systems for multitrack popular songs presents three primary challenges: insufficient fidelity of input conditions, poor structural modeling, and inadequate inter-track harmony learning in generative models. To address these issues, we propose BandCondiNet, a conditional model based on parallel Transformers, designed to process the multiple music sequences and generate high-quality multitrack samples. Specifically, we propose multi-view features across time and instruments as high-fidelity conditions. Moreover, we propose two specialized modules for BandCondiNet: Structure Enhanced Attention (SEA) to strengthen the musical structure, and Cross-Track Transformer (CTT) to enhance inter-track harmony. We conducted both objective and subjective evaluations on two popular music datasets with different sequence lengths. Objective results on the shorter dataset show that BandCondiNet outperforms other conditional models in 9 out of 10 metrics related to fidelity and inference speed, with the exception of Chord Accuracy. On the longer dataset, BandCondiNet surpasses all conditional models across all 10 metrics. Subjective evaluations across four criteria reveal that BandCondiNet trained on the shorter dataset performs best in Richness and performs comparably to state-of-the-art models in the other three criteria, while significantly outperforming them across all criteria when trained on the longer dataset. To further expand the application scope of BandCondiNet, future work should focus on developing an advanced conditional model capable of adapting to more user-friendly input conditions and supporting flexible instrumentation.