Towards Translating Real-World Code with LLMs: A Study of Translating to Rust
Hasan Ferit Eniser, Hanliang Zhang, Cristina David, Meng Wang, Maria, Christakis, Brandon Paulsen, Joey Dodds, Daniel Kroening

TL;DR
This study evaluates the ability of five state-of-the-art LLMs to translate real-world code to Rust, introducing FLOURINE, a tool that verifies translation correctness via differential fuzzing, and analyzing translation success rates and correction capabilities.
Contribution
The paper presents the first comprehensive study of LLM-based code translation to Rust on real-world projects, along with a novel end-to-end translation and verification framework called FLOURINE.
Findings
The best LLM translated 47% of benchmarks successfully.
FLOURINE effectively verifies translation correctness without pre-existing test cases.
Automated feedback strategies improve translation quality and bug fixing.
Abstract
Large language models (LLMs) show promise in code translation - the task of translating code written in one programming language to another language - due to their ability to write code in most programming languages. However, LLM's effectiveness on translating real-world code remains largely unstudied. In this work, we perform the first substantial study on LLM-based translation to Rust by assessing the ability of five state-of-the-art LLMs, GPT4, Claude 3, Claude 2.1, Gemini Pro, and Mixtral. We conduct our study on code extracted from real-world open source projects. To enable our study, we develop FLOURINE, an end-to-end code translation tool that uses differential fuzzing to check if a Rust translation is I/O equivalent to the original source program, eliminating the need for pre-existing test cases. As part of our investigation, we assess both the LLM's ability to produce an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Artificial Intelligence in Law · Translation Studies and Practices
