Pisets: A Robust Speech Recognition System for Lectures and Interviews

TL;DR

Pisets is a robust speech recognition system combining multiple models and advanced techniques to improve transcription accuracy for long, diverse Russian audio recordings, outperforming existing models.

Contribution

The paper introduces a novel three-component architecture with curriculum learning and uncertainty modeling to enhance speech recognition accuracy and robustness.

Findings

Outperforms WhisperX and Whisper in accuracy and robustness

Effective in transcribing long audio across various acoustic conditions

Utilizes curriculum learning and uncertainty modeling for improvements

Abstract

This work presents a speech-to-text system "Pisets" for scientists and journalists which is based on a three-component architecture aimed at improving speech recognition accuracy while minimizing errors and hallucinations associated with the Whisper model. The architecture comprises primary recognition using Wav2Vec2, false positive filtering via the Audio Spectrogram Transformer (AST), and final speech recognition through Whisper. The implementation of curriculum learning methods and the utilization of diverse Russian-language speech corpora significantly enhanced the system's effectiveness. Additionally, advanced uncertainty modeling techniques were introduced, contributing to further improvements in transcription quality. The proposed approaches ensure robust transcribing of long audio data across various acoustic conditions compared to WhisperX and the usual Whisper model. The source code of "Pisets" system is publicly available at GitHub: https://github.com/bond005/pisets.