Sentence Similarity Measures for Fine-Grained Estimation of Topical Relevance in Learner Essays

TL;DR

This paper introduces a new method for assessing sentence relevance in learner essays, improving accuracy by adjusting word embedding weights for better topical relevance estimation.

Contribution

It proposes a novel sentence similarity approach that learns to fine-tune pre-trained word embeddings for enhanced relevance detection in learner writing.

Findings

The new method outperforms existing baselines in accuracy.

Neural embedding-based systems show significant improvements.

Adjusting embedding weights improves topical relevance estimation.

Abstract

We investigate the task of assessing sentence-level prompt relevance in learner essays. Various systems using word overlap, neural embeddings and neural compositional models are evaluated on two datasets of learner writing. We propose a new method for sentence-level similarity calculation, which learns to adjust the weights of pre-trained word embeddings for a specific task, achieving substantially higher accuracy compared to other relevant baselines.