A Study on Lip Localization Techniques used for Lip reading from a Video

TL;DR

This paper reviews various lip localization techniques for lip reading from videos, compares their processing steps, and proposes a new method to improve visual speech recognition especially in noisy or audio-absent scenarios.

Contribution

It provides a comparative analysis of existing lip localization methods and introduces a novel technique to enhance lip reading accuracy in diverse conditions.

Findings

Techniques work on asymmetric lips and with visible teeth or moustache

Lip localization is crucial for visual speech recognition

Proposed method aims to improve lip tracking accuracy

Abstract

In this paper some of the different techniques used to localize the lips from the face are discussed and compared along with its processing steps. Lip localization is the basic step needed to read the lips for extracting visual information from the video input. The techniques could be applied on asymmetric lips and also on the mouth with visible teeth, tongue & mouth with moustache. In the process of Lip reading the following steps are generally used. They are, initially locating lips in the first frame of the video input, then tracking the lips in the following frames using the resulting pixel points of initial step and at last converting the tracked lip model to its corresponding matched letter to give the visual information. A new proposal is also initiated from the discussed techniques. The lip reading is useful in Automatic Speech Recognition when the audio is absent or present low with or without noise in the communication systems. Human Computer communication also will require speech recognition.