A Pilot Study on Mandarin Chinese Cued Speech

TL;DR

This study introduces a novel Mandarin Chinese Cued Speech system that reduces complexity by reclassifying vowels, leveraging French CS similarities, and using head movements for tones, aiming to improve communication for deaf Mandarin speakers.

Contribution

It presents the first official Mandarin Chinese Cued Speech system, combining phonetic coding, vowel reclassification, and cross-linguistic adaptation to enhance speechreading support.

Findings

Achieved over 92% vowel group separability with Gaussian classifier.

Reduced Mandarin finals from 36 to 16 through vowel reclassification.

Demonstrated effective tone coding with head movements.

Abstract

Cued Speech (CS) is a communication system developed for deaf people, which exploits hand cues to complement speechreading at the phonetic level. Currently, it is estimated that CS has been adapted to over 60 languages; however, no official CS system is available for Mandarin Chinese. This article proposes a novel and efficient Mandarin Chinese CS system, satisfying the main criterion that the hand coding constitutes a complement to the lips movements. We propose to code vowels [i, u, y] as semiconsonants when they are followed by other Mandarin finals, which reduces the number of Mandarin finals to be coded from 36 to 16. We establish a coherent similarity between Mandarin Chinese and French vowels for the remaining 16 vowels, which allows us to take advantage of the French CS system. Furthermore, by investigating the lips viseme distribution based on a new corpus, an optimal allocation of the 16 Mandarin vowels to different hand positions is obtained. A Gaussian classifier was used to evaluate the average separability of different allocated vowel groups, which gives 92.08\%, 92.33\%, and 92.73\% for the three speakers, respectively. The consonants are mainly designed according to their similarities with the French CS system, as well as some considerations on the special Mandarin consonants. In our system, the tones of Mandarin are coded with head movements.