AILive Mixer: A Deep Learning based Zero Latency Automatic Music Mixer for Live Music Performances

TL;DR

This paper introduces AILive Mixer, a deep learning system designed for real-time, zero-latency automatic multitrack music mixing tailored for live performances, addressing challenges like acoustic bleed and synchronization.

Contribution

It is the first end-to-end deep learning system specifically developed for live music performance mixing with zero latency constraints.

Findings

Handles acoustic bleed in live multitrack inputs effectively.

Achieves zero-latency processing suitable for live performances.

Predicts mono gains, with potential for future parameter prediction.

Abstract

In this work, we present a deep learning-based automatic multitrack music mixing system catered towards live performances. In a live performance, channels are often corrupted with acoustic bleeds of co-located instruments. Moreover, audio-visual synchronization is of critical importance thus putting a tight constraint on the audio latency. In this work we primarily tackle these two challenges of handling bleeds in the input channels to produce the music mix with zero latency. Although there have been several developments in the field of automatic music mixing in recent times, most or all previous works focus on offline production for isolated instrument signals and to the best of our knowledge, this is the first end-to-end deep learning system developed for live music performances. Our proposed system currently predicts mono gains for a multitrack input, but its design along with the precedent set in past works, allows for easy adaptation to future work of predicting other relevant music mixing parameters.