Generating Adversarial Computer Programs using Optimized Obfuscations

TL;DR

This paper introduces a novel method for generating adversarial computer programs by applying optimized obfuscations, which deceive machine learning models without changing program functionality, and evaluates its effectiveness on Python and Java programs.

Contribution

It presents a general formulation and optimization algorithms for creating adversarial programs through multiple obfuscations, improving attack success and robustness of models against such attacks.

Findings

Achieves 52% improvement over existing attack methods.

Effective in fooling models on Python and Java program summarization.

Enhances training of models to be more robust against adversarial perturbations.

Abstract

Machine learning (ML) models that learn and predict properties of computer programs are increasingly being adopted and deployed. These models have demonstrated success in applications such as auto-completing code, summarizing large programs, and detecting bugs and malware in programs. In this work, we investigate principled ways to adversarially perturb a computer program to fool such learned models, and thus determine their adversarial robustness. We use program obfuscations, which have conventionally been used to avoid attempts at reverse engineering programs, as adversarial perturbations. These perturbations modify programs in ways that do not alter their functionality but can be crafted to deceive an ML model when making a decision. We provide a general formulation for an adversarial program that allows applying multiple obfuscation transformations to a program in any language. We develop first-order optimization algorithms to efficiently determine two key aspects -- which parts of the program to transform, and what transformations to use. We show that it is important to optimize both these aspects to generate the best adversarially perturbed program. Due to the discrete nature of this problem, we also propose using randomized smoothing to improve the attack loss landscape to ease optimization. We evaluate our work on Python and Java programs on the problem of program summarization. We show that our best attack proposal achieves a $52\%$ improvement over a state-of-the-art attack generation approach for programs trained on a seq2seq model. We further show that our formulation is better at training models that are robust to adversarial attacks.