An Attention Ensemble Approach for Efficient Text Classification of Indian Languages

TL;DR

This paper introduces a hybrid CNN-BiLSTM attention ensemble model for efficient text classification of Marathi, an Indian language, achieving state-of-the-art results in technical domain identification tasks.

Contribution

It presents a novel hybrid ensemble approach combining CNN and BiLSTM with attention for resource-constrained Indian language NLP tasks.

Findings

Achieved 89.57% validation accuracy and 0.8875 F1-score.

Outperformed baseline models and other teams in shared task.

Secured the best system submission with 64.26% test accuracy.

Abstract

The recent surge of complex attention-based deep learning architectures has led to extraordinary results in various downstream NLP tasks in the English language. However, such research for resource-constrained and morphologically rich Indian vernacular languages has been relatively limited. This paper proffers team SPPU\_AKAH's solution for the TechDOfication 2020 subtask-1f: which focuses on the coarse-grained technical domain identification of short text documents in Marathi, a Devanagari script-based Indian language. Availing the large dataset at hand, a hybrid CNN-BiLSTM attention ensemble model is proposed that competently combines the intermediate sentence representations generated by the convolutional neural network and the bidirectional long short-term memory, leading to efficient text classification. Experimental results show that the proposed model outperforms various baseline machine learning and deep learning models in the given task, giving the best validation accuracy of 89.57\% and f1-score of 0.8875. Furthermore, the solution resulted in the best system submission for this subtask, giving a test accuracy of 64.26\% and f1-score of 0.6157, transcending the performances of other teams as well as the baseline system given by the organizers of the shared task.