Automated Validation of COBOL to Java Transformation

TL;DR

This paper introduces a framework that uses symbolic execution and automated testing to validate the correctness of COBOL to Java code translations produced by LLMs, ensuring semantic equivalence.

Contribution

It presents a novel validation approach combining symbolic execution and test generation to verify and repair LLM-translated legacy code.

Findings

Successfully generated equivalent JUnit tests for COBOL programs.

Effectively identified translation discrepancies and facilitated code repair.

Enhanced trustworthiness of AI-based code translation tools.

Abstract

Recent advances in Large Language Model (LLM) based Generative AI techniques have made it feasible to translate enterpriselevel code from legacy languages such as COBOL to modern languages such as Java or Python. While the results of LLM-based automatic transformation are encouraging, the resulting code cannot be trusted to correctly translate the original code. We propose a framework and a tool to help validate the equivalence of COBOL and translated Java. The results can also help repair the code if there are some issues and provide feedback to the AI model to improve. We have developed a symbolic-execution-based test generation to automatically generate unit tests for the source COBOL programs which also mocks the external resource calls. We generate equivalent JUnit test cases with equivalent mocking as COBOL and run them to check semantic equivalence between original and translated programs.