E-code: Mastering Efficient Code Generation through Pretrained Models and Expert Encoder Group

TL;DR

E-code is a specialized language model designed for optimizing code efficiency, employing an innovative expert encoder group to significantly outperform existing models in competitive code performance tasks.

Contribution

This paper introduces E-code, a novel program synthesis language model with an expert encoder group, addressing limitations of traditional methods in code efficiency optimization.

Findings

E-code achieved a 54.98% improvement in code efficiency.

The expert encoder group significantly enhances model performance.

Ablation studies validated the importance of model components.

Abstract

Context: With the waning of Moore's Law, the software industry is placing increasing importance on finding alternative solutions for continuous performance enhancement. The significance and research results of software performance optimization have been on the rise in recent years, especially with the advancement propelled by Large Language Models(LLMs). However, traditional strategies for rectifying performance flaws have shown significant limitations at the competitive code efficiency optimization level, and research on this topic is surprisingly scarce. Objective: This study aims to address the research gap in this domain, offering practical solutions to the various challenges encountered. Specifically, we have overcome the constraints of traditional performance error rectification strategies and developed a Language Model (LM) tailored for the competitive code efficiency optimization realm. Method: We introduced E-code, an advanced program synthesis LM. Inspired by the recent success of expert LMs, we designed an innovative structure called the Expert Encoder Group. This structure employs multiple expert encoders to extract features tailored for different input types. We assessed the performance of E-code against other leading models on a competitive dataset and conducted in-depth ablation experiments. Results: Upon systematic evaluation, E-code achieved a 54.98% improvement in code efficiency, significantly outperforming other advanced models. In the ablation experiments, we further validated the significance of the expert encoder group and other components within E-code. Conclusion: The research findings indicate that the expert encoder group can effectively handle various inputs in efficiency optimization tasks, significantly enhancing the model's performance.