# A syllable-character collaborative model for enhanced Pinyin and Chinese recognition

## TL;DR

This paper introduces a new model for Chinese speech recognition that improves accuracy by combining syllables and characters during training.

## Contribution

The novel SCCM model uses phonetic elements and an ensemble approach to reduce recognition errors in Chinese speech.

## Key findings

- The SCCM model reduces pinyin and character error rates compared to prior methods.
- It achieves a 45.7% relative reduction in Character Error Rate on the AISHELL-1 dataset.

## Abstract

In Chinese speech recognition, end-to-end speech recognition models usually use Chinese characters as direct output and perform poorly compared with other language models. The main reason for this phenomenon is that the relationship between Chinese text and pronunciation is more complex. Inspired by the learning process of Chinese beginners, who first master initials, finals, and pinyin before learning characters, we propose the Syllable-Character Collaborative Model (SCCM), which incorporates these phonetic elements into the training process. Additionally, we design a Pinyin-Ensemble module that employs an ensemble learning approach to reduce pinyin recognition errors, which in turn leads to a reduction in text recognition errors. Experiments on AISHELL-1 show that our approach not only reduces pinyin and character error rates compared to a prior end-to-end method using pinyin as auxiliary information, but also achieves a 45.7% relative reduction in Character Error Rate (CER) over the AISHELL-1 baseline.