Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesis

TL;DR

This paper introduces ProgSG, a deep interaction model for program representation that combines code sequences and control data flow graphs to improve machine learning predictions in high-level synthesis for electronic design automation.

Contribution

It proposes a novel deep interaction model for multi-modal program data and a pre-training method based on compiler data flow analysis to enhance design performance prediction.

Findings

ProgSG reduces RMSE of performance predictions by up to 22%.

It achieves 1.10x to 1.26x average performance improvements in design space exploration.

Outperforms existing methods HARP and AutoDSE significantly.

Abstract

In recent years, domain-specific accelerators (DSAs) have gained popularity for applications such as deep learning and autonomous driving. To facilitate DSA designs, programmers use high-level synthesis (HLS) to compile a high-level description written in C/C++ into a design with low-level hardware description languages that eventually synthesize DSAs on circuits. However, creating a high-quality HLS design still demands significant domain knowledge, particularly in microarchitecture decisions expressed as \textit{pragmas}. Thus, it is desirable to automate such decisions with the help of machine learning for predicting the quality of HLS designs, requiring a deeper understanding of the program that consists of original code and pragmas. Naturally, these programs can be considered as sequence data. In addition, these programs can be compiled and converted into a control data flow graph (CDFG). But existing works either fail to leverage both modalities or combine the two in shallow or coarse ways. We propose ProgSG, a model that allows interaction between the source code sequence modality and the graph modality in a deep and fine-grained way. To alleviate the scarcity of labeled designs, a pre-training method is proposed based on a suite of compiler's data flow analysis tasks. Experimental results show that ProgSG reduces the RMSE of design performance predictions by up to $22\%$, and identifies designs with an average of $1.10\times$ and $1.26\times$ (up to $8.17\times$ and $13.31\times$) performance improvement in design space exploration (DSE) task compared to HARP and AutoDSE, respectively.