SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring

TL;DR

This paper introduces SkipFlow, a neural architecture that enhances automatic text scoring by modeling coherence features through relationships between hidden states in an LSTM, achieving state-of-the-art results.

Contribution

The paper proposes a novel SkipFlow mechanism that captures neural coherence features, improving long-sequence understanding in end-to-end automatic text scoring models.

Findings

Achieved state-of-the-art performance on the ASAP dataset.

Outperformed feature engineering and other deep learning models.

Enhanced LSTM with auxiliary neural coherence features.

Abstract

Deep learning has demonstrated tremendous potential for Automatic Text Scoring (ATS) tasks. In this paper, we describe a new neural architecture that enhances vanilla neural network models with auxiliary neural coherence features. Our new method proposes a new \textsc{SkipFlow} mechanism that models relationships between snapshots of the hidden representations of a long short-term memory (LSTM) network as it reads. Subsequently, the semantic relationships between multiple snapshots are used as auxiliary features for prediction. This has two main benefits. Firstly, essays are typically long sequences and therefore the memorization capability of the LSTM network may be insufficient. Implicit access to multiple snapshots can alleviate this problem by acting as a protection against vanishing gradients. The parameters of the \textsc{SkipFlow} mechanism also acts as an auxiliary memory. Secondly, modeling relationships between multiple positions allows our model to learn features that represent and approximate textual coherence. In our model, we call this \textit{neural coherence} features. Overall, we present a unified deep learning architecture that generates neural coherence features as it reads in an end-to-end fashion. Our approach demonstrates state-of-the-art performance on the benchmark ASAP dataset, outperforming not only feature engineering baselines but also other deep learning models.