Characterizing the hyper-parameter space of LSTM language models for mixed context applications

TL;DR

This paper investigates the sensitivity of LSTM language models' hyperparameters when applied to a new code-mixed dataset, highlighting their robustness and implications for reproducibility in real-world applications.

Contribution

The study provides a detailed characterization of LSTM hyper-parameter sensitivity on a novel code-mixed corpus, revealing minimal sensitivity for most parameters.

Findings

Most hyperparameters show minimal sensitivity to the new dataset.

Certain hyperparameters significantly affect model performance.

Results inform best practices for hyper-parameter tuning in real-world scenarios.

Abstract

Applying state of the art deep learning models to novel real world datasets gives a practical evaluation of the generalizability of these models. Of importance in this process is how sensitive the hyper parameters of such models are to novel datasets as this would affect the reproducibility of a model. We present work to characterize the hyper parameter space of an LSTM for language modeling on a code-mixed corpus. We observe that the evaluated model shows minimal sensitivity to our novel dataset bar a few hyper parameters.