ICASSP 2021 Acoustic Echo Cancellation Challenge: Datasets, Testing Framework, and Results

TL;DR

The ICASSP 2021 AEC Challenge provides large real and synthetic datasets, testing frameworks, and results to advance acoustic echo cancellation research in realistic environments, addressing the gap between synthetic and real-world performance.

Contribution

This paper introduces open-source datasets, testing frameworks, and evaluation results to foster research in acoustic echo cancellation under realistic conditions.

Findings

Datasets from over 2,500 real devices and environments are provided.

An online subjective testing framework is available for rapid evaluation.

Challenge winners are selected based on average MOS scores across scenarios.

Abstract

The ICASSP 2021 Acoustic Echo Cancellation Challenge is intended to stimulate research in the area of acoustic echo cancellation (AEC), which is an important part of speech enhancement and still a top issue in audio communication and conferencing systems. Many recent AEC studies report good performance on synthetic datasets where the train and test samples come from the same underlying distribution. However, the AEC performance often degrades significantly on real recordings. Also, most of the conventional objective metrics such as echo return loss enhancement (ERLE) and perceptual evaluation of speech quality (PESQ) do not correlate well with subjective speech quality tests in the presence of background noise and reverberation found in realistic environments. In this challenge, we open source two large datasets to train AEC models under both single talk and double talk scenarios. These datasets consist of recordings from more than 2,500 real audio devices and human speakers in real environments, as well as a synthetic dataset. We open source two large test sets, and we open source an online subjective test framework for researchers to quickly test their results. The winners of this challenge will be selected based on the average Mean Opinion Score (MOS) achieved across all different single talk and double talk scenarios.