Auto-SPT: Automating Semantic Preserving Transformations for Code

TL;DR

Auto-SPT is a framework that automatically generates diverse, semantic-preserving code transformations to improve the robustness of code clone detection models against real-world code variations.

Contribution

It introduces Auto-SPT, a novel LLM-based framework for creating diverse semantic-preserving transformations to enhance code clone detection robustness.

Findings

Auto-SPT produces more diverse transformations than existing methods.

Transformations generated by Auto-SPT significantly reduce clone detector performance.

Auto-SPT can improve training datasets for more robust code clone detection.

Abstract

Machine learning (ML) models for code clone detection determine whether two pieces of code are semantically equivalent, which in turn is a key building block for software-engineering tasks like refactoring and security tasks like vulnerability and malware detection. While these models are predominantly trained on clean, structured code datasets, real-world code often undergoes a variety of semantic-preserving transformations, including refactoring, minification, automated formatting, and compiler optimizations. To address this critical gap between training and test data, we propose Auto-SPT, a novel framework to automatically construct synthetic-data generators for code. Auto-SPT is designed to produce Semantic Preserving Transformations (SPTs) that alter a program's syntactic structure while preserving its functionality and is instantiated on top of Large Language Models (LLMs). In particular, we use LLMs to craft a diverse set of SPTs, generate strong implementations for these SPTs, and compose them to result into strong transformations. Our formal analysis shows that the diversity of SPTs impacts the strength of their composition. We then empirically demonstrate that Auto-SPT generates more diverse SPTs than existing approaches and these SPTs significantly drop the performance of state-of-the-art code clone detectors. Further experiments show Auto-SPT can be used to enhance code datasets for training, to produce code-clone detection models that are robust to real-world, adversarial code transformations.