Visual Words for Automatic Lip-Reading

TL;DR

This paper introduces a new visual words-based approach for automatic lip reading, including novel face and lip localization techniques, aiming to improve visual speech recognition systems.

Contribution

It proposes a novel visual words framework with automatic face and lip localization methods for enhanced lip reading automation.

Findings

Developed a new face localization scheme

Created a lip localization method

Improved accuracy of visual speech recognition

Abstract

Lip reading is used to understand or interpret speech without hearing it, a technique especially mastered by people with hearing difficulties. The ability to lip read enables a person with a hearing impairment to communicate with others and to engage in social activities, which otherwise would be difficult. Recent advances in the fields of computer vision, pattern recognition, and signal processing has led to a growing interest in automating this challenging task of lip reading. Indeed, automating the human ability to lip read, a process referred to as visual speech recognition, could open the door for other novel applications. This thesis investigates various issues faced by an automated lip-reading system and proposes a novel "visual words" based approach to automatic lip reading. The proposed approach includes a novel automatic face localisation scheme and a lip localisation method.