An Analysis of Language Frequency and Error Correction for Esperanto

TL;DR

This paper analyzes Esperanto language frequency and introduces a new error correction dataset, demonstrating GPT-4's superior performance over GPT-3.5 in correcting Esperanto grammar errors.

Contribution

It provides the first comprehensive frequency analysis and a new annotated dataset for Esperanto GEC, and evaluates GPT models' effectiveness in this low-resource language.

Findings

GPT-4 outperforms GPT-3.5 in error correction accuracy

The Eo-GEC dataset enables detailed linguistic error analysis

Advanced language models show promise for low-resource language correction

Abstract

Current Grammar Error Correction (GEC) initiatives tend to focus on major languages, with less attention given to low-resource languages like Esperanto. In this article, we begin to bridge this gap by first conducting a comprehensive frequency analysis using the Eo-GP dataset, created explicitly for this purpose. We then introduce the Eo-GEC dataset, derived from authentic user cases and annotated with fine-grained linguistic details for error identification. Leveraging GPT-3.5 and GPT-4, our experiments show that GPT-4 outperforms GPT-3.5 in both automated and human evaluations, highlighting its efficacy in addressing Esperanto's grammatical peculiarities and illustrating the potential of advanced language models to enhance GEC strategies for less commonly studied languages.