Can General-Purpose Large Language Models Generalize to English-Thai Machine Translation ?

TL;DR

This paper investigates the limitations of large language models in English-Thai machine translation under resource constraints, highlighting the superior performance of specialized models in low-resource settings.

Contribution

It provides a comparative analysis showing that specialized translation models outperform LLMs under strict computational constraints, emphasizing the need for specialized approaches.

Findings

LLMs fail to translate effectively under 4-bit quantization.

Specialized models outperform LLMs with similar or lower computational costs.

Resource constraints significantly impact LLM translation quality.

Abstract

Large language models (LLMs) perform well on common tasks but struggle with generalization in low-resource and low-computation settings. We examine this limitation by testing various LLMs and specialized translation models on English-Thai machine translation and code-switching datasets. Our findings reveal that under more strict computational constraints, such as 4-bit quantization, LLMs fail to translate effectively. In contrast, specialized models, with comparable or lower computational requirements, consistently outperform LLMs. This underscores the importance of specialized models for maintaining performance under resource constraints.