IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator

TL;DR

IMPARA-GED introduces a new reference-free evaluation method for grammatical error correction that leverages a pre-trained language model with enhanced error detection, significantly improving correlation with human judgments.

Contribution

It presents a novel GEC evaluation approach combining grammatical error detection with quality estimation, outperforming existing methods in correlation with human assessments.

Findings

Achieves highest correlation with human evaluations on SEEDA dataset.

Utilizes pre-trained language models with enhanced GED capabilities.

Demonstrates effectiveness of reference-free GEC evaluation.

Abstract

We propose IMPARA-GED, a novel reference-free automatic grammatical error correction (GEC) evaluation method with grammatical error detection (GED) capabilities. We focus on the quality estimator of IMPARA, an existing automatic GEC evaluation method, and construct that of IMPARA-GED using a pre-trained language model with enhanced GED capabilities. Experimental results on SEEDA, a meta-evaluation dataset for automatic GEC evaluation methods, demonstrate that IMPARA-GED achieves the highest correlation with human sentence-level evaluations.