Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, Shuming Shi,, Zhaopeng Tu

TL;DR
This paper evaluates ChatGPT's machine translation capabilities, finding that with GPT-4, it performs competitively with commercial systems, especially when using pivot prompting for distant languages, and is considered a good translator.
Contribution
The study provides a comprehensive evaluation of ChatGPT's translation performance, introduces pivot prompting to improve distant language translation, and demonstrates GPT-4's significant performance boost.
Findings
ChatGPT performs well on high-resource European languages.
GPT-4 significantly improves translation quality, making ChatGPT comparable to commercial systems.
Pivot prompting enhances translation accuracy for distant languages.
Abstract
This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the prompts advised by ChatGPT to trigger its translation ability and find that the candidate prompts generally work well with minor performance differences. By evaluating on a number of benchmark test sets, we find that ChatGPT performs competitively with commercial translation products (e.g., Google Translate) on high-resource European languages but lags behind significantly on low-resource or distant languages. As for the translation robustness, ChatGPT does not perform as well as the commercial systems on biomedical abstracts or Reddit comments but exhibits good results on spoken language. Further, we explore an interesting strategy named for distant languages, which asks ChatGPT to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Topic Modeling · Explainable Artificial Intelligence (XAI)
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · {Dispute@FaQ-s}How to file a dispute with Expedia? · Multi-Head Attention · 15 Ways to Contact How can i speak to someone at Delta Airlines · Attention Is All You Need · Cosine Annealing · Linear Layer · Softmax · Linear Warmup With Cosine Annealing · Dropout
