An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM

TL;DR

This paper introduces an ensemble question classification model that combines Electra, GloVe, and LSTM to improve accuracy on the TREC dataset, demonstrating the benefits of integrating diverse NLP techniques.

Contribution

The study presents a novel ensemble approach that effectively combines transformer, word embedding, and sequence learning models for question classification.

Findings

Achieved 80% accuracy on TREC dataset

Outperformed individual models like BERT and RoBERTa

Proved the effectiveness of model integration for NLP tasks

Abstract

Natural Language Processing (NLP) has emerged as a crucial technology for understanding and generating human language, playing an essential role in tasks such as machine translation, sentiment analysis, and more pertinently, question classification. As a subfield within NLP, question classification focuses on determining the type of information being sought, a fundamental step for downstream applications like question answering systems. This study presents an innovative ensemble approach for question classification, combining the strengths of Electra, GloVe, and LSTM models. Rigorously tested on the well-regarded TREC dataset, the model demonstrates how the integration of these disparate technologies can lead to superior results. Electra brings in its transformer-based capabilities for complex language understanding, GloVe offers global vector representations for capturing word-level semantics, and LSTM contributes its sequence learning abilities to model long-term dependencies. By fusing these elements strategically, our ensemble model delivers a robust and efficient solution for the complex task of question classification. Through rigorous comparisons with well-known models like BERT, RoBERTa, and DistilBERT, the ensemble approach verifies its effectiveness by attaining an 80% accuracy score on the test dataset.