Native Language Identification using Stacked Generalization

TL;DR

This paper systematically evaluates ensemble and stacking methods for Native Language Identification, achieving state-of-the-art results and introducing statistical significance testing to compare models.

Contribution

It provides a comprehensive analysis of ensemble architectures for NLI, including the first application of significance testing in this context.

Findings

State-of-the-art accuracy on three datasets

Meta-classification models outperform previous methods

Statistical significance confirms improvements

Abstract

Ensemble methods using multiple classifiers have proven to be the most successful approach for the task of Native Language Identification (NLI), achieving the current state of the art. However, a systematic examination of ensemble methods for NLI has yet to be conducted. Additionally, deeper ensemble architectures such as classifier stacking have not been closely evaluated. We present a set of experiments using three ensemble-based models, testing each with multiple configurations and algorithms. This includes a rigorous application of meta-classification models for NLI, achieving state-of-the-art results on three datasets from different languages. We also present the first use of statistical significance testing for comparing NLI systems, showing that our results are significantly better than the previous state of the art. We make available a collection of test set predictions to facilitate future statistical tests.