Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddings

TL;DR

This paper explores leveraging compiler intermediate representations and genetic algorithms to improve neural program embeddings for better program analysis tasks.

Contribution

It introduces methods to enhance embeddings using LLVM IR and optimization flag sequences identified by genetic algorithms.

Findings

Embedding quality improves with IR-based training.

Genetic algorithms can find near-optimal optimization flags.

Enhanced embeddings benefit program analysis tasks.

Abstract

Neural program embeddings have demonstrated considerable promise in a range of program analysis tasks, including clone identification, program repair, code completion, and program synthesis. However, most existing methods generate neural program embeddings directly from the program source codes, by learning from features such as tokens, abstract syntax trees, and control flow graphs. This paper takes a fresh look at how to improve program embeddings by leveraging compiler intermediate representation (IR). We first demonstrate simple yet highly effective methods for enhancing embedding quality by training embedding models alongside source code and LLVM IR generated by default optimization levels (e.g., -O2). We then introduce IRGen, a framework based on genetic algorithms (GA), to identify (near-)optimal sequences of optimization flags that can significantly improve embedding quality.