Manifold-Kernels Comparison in MKPLS for Visual Speech Recognition

TL;DR

This paper evaluates various manifold kernels within the MKPLS framework to improve visual speech recognition accuracy by effectively measuring distances between visual units on manifolds.

Contribution

It provides a theoretical comparison and empirical evaluation of multiple manifold kernels for visual speech recognition using MKPLS.

Findings

Certain kernels outperform others in recognition accuracy

Manifold kernels significantly improve distance measurement in visual speech tasks

The framework facilitates exploration of different kernels for better performance

Abstract

Speech recognition is a challenging problem. Due to the acoustic limitations, using visual information is essential for improving the recognition accuracy in real-life unconstraint situations. One common approach is to model the visual recognition as nonlinear optimization problem. Measuring the distances between visual units is essential for solving this problem. Embedding the visual units on a manifold and using manifold kernels is one way to measure these distances. This work is intended to evaluate the performance of several manifold kernels for solving the problem of visual speech recognition. We show the theory behind each kernel. We apply manifold kernel partial least squares framework to OuluVs and AvLetters databases, and show empirical comparison between all kernels. This framework provides convenient way to explore different kernels.