Improving the Generalization Ability in Essay Coherence Evaluation through Monotonic Constraints

TL;DR

This paper introduces a coherence scoring model for essays that uses monotonic constraints to improve generalization, combining local coherence and punctuation features, and demonstrates competitive results in NLPCC 2023 shared tasks.

Contribution

The paper proposes a novel coherence evaluation model with monotonicity constraints on features, enhancing generalization in essay coherence assessment.

Findings

Achieved third place in NLPCC 2023 track 1

Second place in track 2

First place in tracks 3 and 4

Abstract

Coherence is a crucial aspect of evaluating text readability and can be assessed through two primary factors when evaluating an essay in a scoring scenario. The first factor is logical coherence, characterized by the appropriate use of discourse connectives and the establishment of logical relationships between sentences. The second factor is the appropriateness of punctuation, as inappropriate punctuation can lead to confused sentence structure. To address these concerns, we propose a coherence scoring model consisting of a regression model with two feature extractors: a local coherence discriminative model and a punctuation correction model. We employ gradient-boosting regression trees as the regression model and impose monotonicity constraints on the input features. The results show that our proposed model better generalizes unseen data. The model achieved third place in track 1 of NLPCC 2023 shared task 7. Additionally, we briefly introduce our solution for the remaining tracks, which achieves second place for track 2 and first place for both track 3 and track 4.