"This is Houston. Say again, please". The Behavox system for the Apollo-11 Fearless Steps Challenge (phase II)

TL;DR

This paper presents the Behavox system's approach and results for speech activity detection, speaker diarization, and automatic speech recognition in the NASA Apollo-11 challenge, demonstrating significant improvements and first-place rankings.

Contribution

The paper introduces semi-supervised training using a large unlabeled Apollo-11 corpus and compares multiple SAD and SD systems for challenging long recordings.

Findings

Over 17% relative WER reduction with semi-supervised training

Achieved first place in SD and ASR tasks

Substantial performance improvements over baseline systems

Abstract

We describe the speech activity detection (SAD), speaker diarization (SD), and automatic speech recognition (ASR) experiments conducted by the Behavox team for the Interspeech 2020 Fearless Steps Challenge (FSC-2). A relatively small amount of labeled data, a large variety of speakers and channel distortions, specific lexicon and speaking style resulted in high error rates on the systems which involved this data. In addition to approximately 36 hours of annotated NASA mission recordings, the organizers provided a much larger but unlabeled 19k hour Apollo-11 corpus that we also explore for semi-supervised training of ASR acoustic and language models, observing more than 17% relative word error rate improvement compared to training on the FSC-2 data only. We also compare several SAD and SD systems to approach the most difficult tracks of the challenge (track 1 for diarization and ASR), where long 30-minute audio recordings are provided for evaluation without segmentation or speaker information. For all systems, we report substantial performance improvements compared to the FSC-2 baseline systems, and achieved a first-place ranking for SD and ASR and fourth-place for SAD in the challenge.