CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challenge

TL;DR

The CNVSRC 2023 challenge evaluated large vocabulary continuous visual speech recognition for Chinese, demonstrating significant improvements over baselines in single- and multi-speaker tasks and reviewing effective techniques.

Contribution

First Chinese continuous visual speech recognition challenge providing comprehensive evaluation and analysis of techniques for single- and multi-speaker tasks.

Findings

Best system outperformed baseline significantly

Single-speaker task achieved higher accuracy

Effective techniques identified for visual speech recognition

Abstract

The first Chinese Continuous Visual Speech Recognition Challenge aimed to probe the performance of Large Vocabulary Continuous Visual Speech Recognition (LVC-VSR) on two tasks: (1) Single-speaker VSR for a particular speaker and (2) Multi-speaker VSR for a set of registered speakers. The challenge yielded highly successful results, with the best submission significantly outperforming the baseline, particularly in the single-speaker task. This paper comprehensively reviews the challenge, encompassing the data profile, task specifications, and baseline system construction. It also summarises the representative techniques employed by the submitted systems, highlighting the most effective approaches. Additional information and resources about this challenge can be accessed through the official website at http://cnceleb.org/competition.