An automatic mixing speech enhancement system for multi-track audio

TL;DR

This paper introduces an automatic speech enhancement system for multitrack audio that reduces auditory masking and improves clarity in multi-speaker scenarios, evaluated through perceptual quality metrics and listening tests.

Contribution

It presents a novel iterative harmony search-based method for applying audio effects to minimize masking, outperforming existing auto-mixing systems and rivaling professional mixes.

Findings

System effectively reduces auditory masking.

Outperforms existing auto-mixing systems in listening tests.

Achieves comparable quality to professional sound engineers' mixes.

Abstract

We propose a speech enhancement system for multitrack audio. The system will minimize auditory masking while allowing one to hear multiple simultaneous speakers. The system can be used in multiple communication scenarios e.g., teleconferencing, invoice gaming, and live streaming. The ITU-R BS.1387 Perceptual Evaluation of Audio Quality (PEAQ) model is used to evaluate the amount of masking in the audio signals. Different audio effects e.g., level balance, equalization, dynamic range compression, and spatialization are applied via an iterative Harmony searching algorithm that aims to minimize the masking. In the subjective listening test, the designed system can compete with mixes by professional sound engineers and outperforms mixes by existing auto-mixing systems.