How to Design Translation Prompts for ChatGPT: An Empirical Study

TL;DR

This study empirically demonstrates that with well-designed prompts, ChatGPT can outperform commercial translation systems in high-resource and domain-specific language translations, showing significant potential for translation tasks.

Contribution

It introduces effective translation prompts for ChatGPT and provides empirical evidence of its superior performance in various translation scenarios.

Findings

ChatGPT with prompts outperforms commercial systems in high-resource translations.

ChatGPT adapts well to domain-specific translation tasks.

Few-shot prompts improve translation quality consistently.

Abstract

The recently released ChatGPT has demonstrated surprising abilities in natural language understanding and natural language generation. Machine translation relies heavily on the abilities of language understanding and generation. Thus, in this paper, we explore how to assist machine translation with ChatGPT. We adopt several translation prompts on a wide range of translations. Our experimental results show that ChatGPT with designed translation prompts can achieve comparable or better performance over commercial translation systems for high-resource language translations. We further evaluate the translation quality using multiple references, and ChatGPT achieves superior performance compared to commercial systems. We also conduct experiments on domain-specific translations, the final results show that ChatGPT is able to comprehend the provided domain keyword and adjust accordingly to output proper translations. At last, we perform few-shot prompts that show consistent improvement across different base prompts. Our work provides empirical evidence that ChatGPT still has great potential in translations.