CNVSRC 2024: The Second Chinese Continuous Visual Speech Recognition Challenge

TL;DR

CNVSRC 2024 is a challenge that advances Chinese continuous visual speech recognition by introducing stronger baselines, new datasets, and innovative methods, aiming to improve accuracy and robustness in real-world scenarios.

Contribution

The paper introduces CNVSRC 2024 with enhanced datasets, improved baseline systems, and novel techniques to push the boundaries of Chinese LVC-VSR research.

Findings

Demonstrated improved recognition accuracy over previous challenge.

Introduced new datasets to enhance data diversity and volume.

Showcased innovative data processing and model training strategies.

Abstract

This paper presents the second Chinese Continuous Visual Speech Recognition Challenge (CNVSRC 2024), which builds on CNVSRC 2023 to advance research in Chinese Large Vocabulary Continuous Visual Speech Recognition (LVC-VSR). The challenge evaluates two test scenarios: reading in recording studios and Internet speech. CNVSRC 2024 uses the same datasets as its predecessor CNVSRC 2023, which involves CN-CVS for training and CNVSRC-Single/Multi for development and evaluation. However, CNVSRC 2024 introduced two key improvements: (1) a stronger baseline system, and (2) an additional dataset, CN-CVS2-P1, for open tracks to improve data volume and diversity. The new challenge has demonstrated several important innovations in data preprocessing, feature extraction, model design, and training strategies, further pushing the state-of-the-art in Chinese LVC-VSR. More details and resources are available at the official website.