PERFOGRAPH: A Numerical Aware Program Graph Representation for Performance Optimization and Program Analysis

TL;DR

PERFOGRAPH is a novel graph-based program representation that incorporates numerical and data structure information, significantly improving machine learning-based program analysis and optimization tasks.

Contribution

It introduces a scalable, flexible program graph representation with numerical awareness and an adapted embedding method, outperforming existing methods in multiple benchmarks.

Findings

Reduces error rate by 7.4% on AMD dataset

Achieves 10% error reduction on NVIDIA dataset

Sets new state-of-the-art results in performance optimization tasks

Abstract

The remarkable growth and significant success of machine learning have expanded its applications into programming languages and program analysis. However, a key challenge in adopting the latest machine learning methods is the representation of programming languages, which directly impacts the ability of machine learning methods to reason about programs. The absence of numerical awareness, aggregate data structure information, and improper way of presenting variables in previous representation works have limited their performances. To overcome the limitations and challenges of current program representations, we propose a graph-based program representation called PERFOGRAPH. PERFOGRAPH can capture numerical information and the aggregate data structure by introducing new nodes and edges. Furthermore, we propose an adapted embedding method to incorporate numerical awareness. These enhancements make PERFOGRAPH a highly flexible and scalable representation that effectively captures programs intricate dependencies and semantics. Consequently, it serves as a powerful tool for various applications such as program analysis, performance optimization, and parallelism discovery. Our experimental results demonstrate that PERFOGRAPH outperforms existing representations and sets new state-of-the-art results by reducing the error rate by 7.4% (AMD dataset) and 10% (NVIDIA dataset) in the well-known Device Mapping challenge. It also sets new state-of-the-art results in various performance optimization tasks like Parallelism Discovery and NUMA and Prefetchers Configuration prediction.