Classifier Combination Approach for Question Classification for Bengali Question Answering System

TL;DR

This paper demonstrates that combining multiple classifiers using ensemble, stacking, and voting techniques improves question classification accuracy in Bengali, especially with a two-layer taxonomy, outperforming individual models.

Contribution

The study introduces a multi-model combination approach for Bengali question classification, achieving higher accuracy than single classifiers and extending taxonomy for finer classification.

Findings

Classifier combination improves accuracy by 4.02%.

Stacking achieves 87.79% accuracy on fine-grained classification.

Approach can be adapted for other Indo-Aryan languages.

Abstract

Question classification (QC) is a prime constituent of automated question answering system. The work presented here demonstrates that the combination of multiple models achieve better classification performance than those obtained with existing individual models for the question classification task in Bengali. We have exploited state-of-the-art multiple model combination techniques, i.e., ensemble, stacking and voting, to increase QC accuracy. Lexical, syntactic and semantic features of Bengali questions are used for four well-known classifiers, namely Na\"{\i}ve Bayes, kernel Na\"{\i}ve Bayes, Rule Induction, and Decision Tree, which serve as our base learners. Single-layer question-class taxonomy with 8 coarse-grained classes is extended to two-layer taxonomy by adding 69 fine-grained classes. We carried out the experiments both on single-layer and two-layer taxonomies. Experimental results confirmed that classifier combination approaches outperform single classifier classification approaches by 4.02% for coarse-grained question classes. Overall, the stacking approach produces the best results for fine-grained classification and achieves 87.79% of accuracy. The approach presented here could be used in other Indo-Aryan or Indic languages to develop a question answering system.