Enhancing Deployment-Time Predictive Model Robustness for Code Analysis and Optimization

TL;DR

This paper presents Prom, an open-source library that enhances the robustness of deployment-time predictive models in code analysis by identifying and relabeling mispredicted samples, significantly improving accuracy under changing conditions.

Contribution

The paper introduces Prom, a novel tool that uses statistical assessments and feedback to improve model robustness during deployment in code analysis tasks.

Findings

Prom identifies up to 100% of mispredictions.

Relabeling 5% of identified samples improves model performance to training levels.

Prom is effective across 13 models and 5 code analysis tasks.

Abstract

Supervised machine learning techniques have shown promising results in code analysis and optimization problems. However, a learning-based solution can be brittle because minor changes in hardware or application workloads -- such as facing a new CPU architecture or code pattern -- may jeopardize decision accuracy, ultimately undermining model robustness. We introduce Prom, an open-source library to enhance the robustness and performance of predictive models against such changes during deployment. Prom achieves this by using statistical assessments to identify test samples prone to mispredictions and using feedback on these samples to improve a deployed model. We showcase Prom by applying it to 13 representative machine learning models across 5 code analysis and optimization tasks. Our extensive evaluation demonstrates that Prom can successfully identify an average of 96% (up to 100%) of mispredictions. By relabeling up to 5% of the Prom-identified samples through incremental learning, Prom can help a deployed model achieve a performance comparable to that attained during its model training phase.