Definition of Visual Speech Element and Research on a Method of Extracting Feature Vector for Korean Lip-Reading

TL;DR

This paper defines Korean visemes, proposes a method to extract 20-dimensional visual feature vectors combining static and dynamic features, and evaluates word recognition efficiency using a 3-viseme HMM.

Contribution

It introduces a new viseme definition for Korean and a feature extraction method that enhances lip-reading accuracy.

Findings

Effective viseme definitions for Korean

Successful extraction of combined static and dynamic features

Improved word recognition accuracy with 3-viseme HMM

Abstract

In this paper, we defined the viseme (visual speech element) and described about the method of extracting visual feature vector. We defined the 10 visemes based on vowel by analyzing of Korean utterance and proposed the method of extracting the 20-dimensional visual feature vector, combination of static features and dynamic features. Lastly, we took an experiment in recognizing words based on 3-viseme HMM and evaluated the efficiency.