Language Semantics Interpretation with an Interaction-based Recurrent Neural Networks

TL;DR

This paper introduces a new influence score, a greedy search algorithm, and a feature engineering technique to enhance text classification by better capturing language semantics and dependencies.

Contribution

It presents novel methods for interpreting language semantics in neural networks, improving prediction accuracy and understanding of text classification models.

Findings

81% error reduction in IMDB classification task

Effective detection of important language semantics

Enhanced handling of long-term dependencies

Abstract

Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models is capable making good predictions yet there is lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), a greedy search algorithm called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the "dagger technique". First, the paper proposes a novel influence score (I-score) to detect and search for the important language semantics in text document that are useful for making good prediction in text classification tasks. Next, a greedy search algorithm called the Backward Dropping Algorithm is proposed to handle long-term dependencies in the dataset. Moreover, the paper proposes a novel engineering technique called the "dagger technique" that fully preserve the relationship between explanatory variable and response variable. The proposed techniques can be further generalized into any feed-forward Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and any neural network. A real-world application on the Internet Movie Database (IMDB) is used and the proposed methods are applied to improve prediction performance with an 81% error reduction comparing with other popular peers if I-score and "dagger technique" are not implemented.