Language identification as improvement for lip-based biometric visual systems

TL;DR

This study explores using linguistic information as a soft biometric trait to improve lip-based visual identification systems, demonstrating significant performance enhancements through data fusion and machine learning methods.

Contribution

It introduces a novel approach integrating spoken language as a biometric trait to enhance lip-based identification accuracy, supported by a new multilingual dataset.

Findings

Significant improvement in identification performance with linguistic data integration

Effective score-based fusion strategy enhances system accuracy

Evaluation of deep and machine learning methods for language recognition

Abstract

Language has always been one of humanity's defining characteristics. Visual Language Identification (VLI) is a relatively new field of research that is complex and largely understudied. In this paper, we present a preliminary study in which we use linguistic information as a soft biometric trait to enhance the performance of a visual (auditory-free) identification system based on lip movement. We report a significant improvement in the identification performance of the proposed visual system as a result of the integration of these data using a score-based fusion strategy. Methods of Deep and Machine Learning are considered and evaluated. To the experimentation purposes, the dataset called laBial Articulation for the proBlem of the spokEn Language rEcognition (BABELE), consisting of eight different languages, has been created. It includes a collection of different features of which the spoken language represents the most relevant, while each sample is also manually labelled with gender and age of the subjects.