Some Grammatical Errors are Frequent, Others are Important

TL;DR

This paper investigates the varying importance of different grammatical error types in correction systems, revealing that some rare errors are more disturbing to humans than common ones, impacting evaluation and improvement strategies.

Contribution

It introduces a method to quantify the importance of grammatical error types to humans, highlighting the need to consider error significance beyond frequency.

Findings

Some rare errors are more disturbing than common errors.

Error importance varies significantly across error types.

Evaluation metrics should account for error importance, not just frequency.

Abstract

In Grammatical Error Correction, systems are evaluated by the number of errors they correct. However, no one has assessed whether all error types are equally important. We provide and apply a method to quantify the importance of different grammatical error types to humans. We show that some rare errors are considered disturbing while other common ones are not. This affects possible directions to improve both systems and their evaluation.