Fine-grained Language Identification with Multilingual CapsNet Model

TL;DR

This paper introduces a real-time, fine-grained spoken language identification system using a novel Multilingual CapsNet architecture operating on spectrograms, achieving 91.8% accuracy with minimal data and pre-processing.

Contribution

It proposes a new CapsNet-based model for language identification that outperforms previous RNN and iVector approaches in accuracy and efficiency.

Findings

Achieved 91.8% accuracy in 5-second audio clips.

CapsNet architecture effectively captures language features from spectrograms.

Minimal data and pre-processing required for high accuracy.

Abstract

Due to a drastic improvement in the quality of internet services worldwide, there is an explosion of multilingual content generation and consumption. This is especially prevalent in countries with large multilingual audience, who are increasingly consuming media outside their linguistic familiarity/preference. Hence, there is an increasing need for real-time and fine-grained content analysis services, including language identification, content transcription, and analysis. Accurate and fine-grained spoken language detection is an essential first step for all the subsequent content analysis algorithms. Current techniques in spoken language detection may lack on one of these fronts: accuracy, fine-grained detection, data requirements, manual effort in data collection \& pre-processing. Hence in this work, a real-time language detection approach to detect spoken language from 5 seconds' audio clips with an accuracy of 91.8\% is presented with exiguous data requirements and minimal pre-processing. Novel architectures for Capsule Networks is proposed which operates on spectrogram images of the provided audio snippets. We use previous approaches based on Recurrent Neural Networks and iVectors to present the results. Finally we show a ``Non-Class'' analysis to further stress on why CapsNet architecture works for LID task.