AISHELL-5: The First Open-Source In-Car Multi-Channel Multi-Speaker Speech Dataset for Automatic Speech Diarization and Recognition

TL;DR

AISHELL-5 is the first open-source multi-channel in-car Mandarin speech dataset, enabling research on automatic speech recognition and diarization in complex driving environments with real-world noise and multi-speaker scenarios.

Contribution

This paper introduces AISHELL-5, the first comprehensive open-source in-car multi-channel Mandarin speech dataset with real driving scenarios and noise, along with a baseline system for speech separation and recognition.

Findings

Mainstream ASR models face challenges on AISHELL-5 data.

The dataset provides a new benchmark for in-car speech recognition.

Baseline system demonstrates the feasibility of multi-speaker in-car ASR.

Abstract

This paper delineates AISHELL-5, the first open-source in-car multi-channel multi-speaker Mandarin automatic speech recognition (ASR) dataset. AISHLL-5 includes two parts: (1) over 100 hours of multi-channel speech data recorded in an electric vehicle across more than 60 real driving scenarios. This audio data consists of four far-field speech signals captured by microphones located on each car door, as well as near-field signals obtained from high-fidelity headset microphones worn by each speaker. (2) a collection of 40 hours of real-world environmental noise recordings, which supports the in-car speech data simulation. Moreover, we also provide an open-access, reproducible baseline system based on this dataset. This system features a speech frontend model that employs speech source separation to extract each speaker's clean speech from the far-field signals, along with a speech recognition module that accurately transcribes the content of each individual speaker. Experimental results demonstrate the challenges faced by various mainstream ASR models when evaluated on the AISHELL-5. We firmly believe the AISHELL-5 dataset will significantly advance the research on ASR systems under complex driving scenarios by establishing the first publicly available in-car ASR benchmark.