Exploring the Feasibility of Multilingual Grammatical Error Correction with a Single LLM up to 9B parameters: A Comparative Study of 17 Models

TL;DR

This study evaluates 17 multilingual language models, including Gemma 9B, for grammatical error correction across English, German, Italian, and Swedish, highlighting the best performers and common issues.

Contribution

It provides a comparative analysis of 17 models for multilingual grammatical error correction, identifying the top-performing model Gemma 9B and analyzing common challenges.

Findings

Gemma 9B outperforms other models in all four languages.

Six models improve grammatical correctness across languages.

Models tend to make small, targeted corrections.

Abstract

Recent language models can successfully solve various language-related tasks, and many understand inputs stated in different languages. In this paper, we explore the performance of 17 popular models used to correct grammatical issues in texts stated in English, German, Italian, and Swedish when using a single model to correct texts in all those languages. We analyze the outputs generated by these models, focusing on decreasing the number of grammatical errors while keeping the changes small. The conclusions drawn help us understand what problems occur among those models and which models can be recommended for multilingual grammatical error correction tasks. We list six models that improve grammatical correctness in all four languages and show that Gemma 9B is currently the best performing one for the languages considered.