Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects

TL;DR

This paper introduces a contrastive learning-based system for music mixing style transfer that effectively disentangles audio effects and converts mixing styles from reference recordings, validated through objective and subjective evaluations.

Contribution

It presents a novel end-to-end style transfer method using a contrastive pre-trained encoder trained in a self-supervised manner, addressing data scarcity issues.

Findings

Successfully transfers mixing style close to reference

Encoder effectively disentangles audio effects

System is robust with source separation integration

Abstract

We propose an end-to-end music mixing style transfer system that converts the mixing style of an input multitrack to that of a reference song. This is achieved with an encoder pre-trained with a contrastive objective to extract only audio effects related information from a reference music recording. All our models are trained in a self-supervised manner from an already-processed wet multitrack dataset with an effective data preprocessing method that alleviates the data scarcity of obtaining unprocessed dry data. We analyze the proposed encoder for the disentanglement capability of audio effects and also validate its performance for mixing style transfer through both objective and subjective evaluations. From the results, we show the proposed system not only converts the mixing style of multitrack audio close to a reference but is also robust with mixture-wise style transfer upon using a music source separation model.