Visual Speech Recognition

TL;DR

Visual speech recognition automates lip reading using computer vision and AI, enabling applications like HCI, speaker recognition, and sign language interpretation, especially aiding those with hearing impairments.

Contribution

This paper reviews recent advances in automating lip reading through visual speech recognition, highlighting its techniques and potential applications.

Findings

Significant progress in AI-based lip reading techniques

Enhanced accuracy in visual speech recognition systems

Potential for diverse applications like HCI and surveillance

Abstract

Lip reading is used to understand or interpret speech without hearing it, a technique especially mastered by people with hearing difficulties. The ability to lip read enables a person with a hearing impairment to communicate with others and to engage in social activities, which otherwise would be difficult. Recent advances in the fields of computer vision, pattern recognition, and signal processing has led to a growing interest in automating this challenging task of lip reading. Indeed, automating the human ability to lip read, a process referred to as visual speech recognition (VSR) (or sometimes speech reading), could open the door for other novel related applications. VSR has received a great deal of attention in the last decade for its potential use in applications such as human-computer interaction (HCI), audio-visual speech recognition (AVSR), speaker recognition, talking heads, sign language recognition and video surveillance. Its main aim is to recognise spoken word(s) by using only the visual signal that is produced during speech. Hence, VSR deals with the visual domain of speech and involves image processing, artificial intelligence, object detection, pattern recognition, statistical modelling, etc.