Modeling Structural Similarities between Documents for Coherence Assessment with Graph Convolutional Networks

TL;DR

This paper introduces a GCN-based model that captures structural similarities between documents to improve coherence assessment and essay scoring, outperforming existing methods.

Contribution

It proposes a novel graph convolutional network approach that models inter-document relationships through shared subgraph patterns, enhancing coherence evaluation.

Findings

Outperforms all baseline methods on coherence assessment

Achieves state-of-the-art results in automated essay scoring

Effectively captures structural similarities between documents

Abstract

Coherence is an important aspect of text quality, and various approaches have been applied to coherence modeling. However, existing methods solely focus on a single document's coherence patterns, ignoring the underlying correlation between documents. We investigate a GCN-based coherence model that is capable of capturing structural similarities between documents. Our model first creates a graph structure for each document, from where we mine different subgraph patterns. We then construct a heterogeneous graph for the training corpus, connecting documents based on their shared subgraphs. Finally, a GCN is applied to the heterogeneous graph to model the connectivity relationships. We evaluate our method on two tasks, assessing discourse coherence and automated essay scoring. Results show that our GCN-based model outperforms all baselines, achieving a new state-of-the-art on both tasks.