gec-metrics: A Unified Library for Grammatical Error Correction Evaluation

TL;DR

gec-metrics is a comprehensive, extensible library that standardizes and simplifies the evaluation of grammatical error correction systems, promoting fair comparisons and facilitating metric development.

Contribution

It provides a unified, API-focused platform for GEC evaluation metrics, including meta-evaluation and visualization tools, enhancing consistency and extensibility in the field.

Findings

Ensures consistent GEC system evaluations

Supports development of new GEC metrics

Includes analysis and visualization tools

Abstract

We introduce gec-metrics, a library for using and developing grammatical error correction (GEC) evaluation metrics through a unified interface. Our library enables fair system comparisons by ensuring that everyone conducts evaluations using a consistent implementation. Moreover, it is designed with a strong focus on API usage, making it highly extensible. It also includes meta-evaluation functionalities and provides analysis and visualization scripts, contributing to developing GEC evaluation metrics. Our code is released under the MIT license and is also distributed as an installable package. The video is available on YouTube.