End-to-End Language Identification using Multi-Head Self-Attention and 1D Convolutional Neural Networks

TL;DR

This paper introduces a novel end-to-end language identification system for Indian languages that combines multi-head self-attention with 1D convolutional neural networks and raw waveform input, achieving high accuracy.

Contribution

The work presents a new model architecture integrating multi-head self-attention with raw waveform 1D CNNs for language ID, outperforming baseline models and handcrafted feature-based approaches.

Findings

Achieved 95.90% F1-score on Indian language dataset.

Outperformed baseline by 3.69% in F1-score.

Raw waveform models improved performance by 1.7%.

Abstract

In this work, we propose a new approach for language identification using multi-head self-attention combined with raw waveform based 1D convolutional neural networks for Indian languages. Our approach uses an encoder, multi-head selfattention, and a statistics pooling layer. The encoder learns features directly from raw waveforms using 1D convolution kernels and an LSTM layer. The LSTM layer captures temporal information between the features extracted by the 1D convolutional layer. The multi-head self-attention layer takes outputs of the LSTM layer and applies self-attention mechanisms on these features with M different heads. This process helps the model give more weightage to the more useful features and less weightage to the less relevant features. Finally, the frame-level features are combined using a statistics pooling layer to extract the utterance-level feature vector label prediction. We conduct all our experiments on the 373 hrs of audio data for eight different Indian languages. Our experiments show that our approach outperforms the baseline model by an absolute 3.69% improvement in F1-score and achieves the best F1-score of 95.90%. Our approach also shows that using raw waveform models gets a 1.7% improvement in performance compared to the models built using handcrafted features.