STRATA: Simple, Gradient-Free Attacks for Models of Code

TL;DR

This paper introduces STRATA, a simple gradient-free method for generating adversarial examples on code models, exploiting token frequency and embedding relationships to achieve state-of-the-art results efficiently.

Contribution

The paper presents a novel, gradient-free approach for creating adversarial examples in code models, leveraging token frequency and embedding norms.

Findings

STRATA outperforms gradient-based methods in effectiveness.

The method requires less computational resources.

It maintains code functionality while fooling models.

Abstract

Neural networks are well-known to be vulnerable to imperceptible perturbations in the input, called adversarial examples, that result in misclassification. Generating adversarial examples for source code poses an additional challenge compared to the domains of images and natural language, because source code perturbations must retain the functional meaning of the code. We identify a striking relationship between token frequency statistics and learned token embeddings: the L2 norm of learned token embeddings increases with the frequency of the token except for the highest-frequnecy tokens. We leverage this relationship to construct a simple and efficient gradient-free method for generating state-of-the-art adversarial examples on models of code. Our method empirically outperforms competing gradient-based methods with less information and less computational effort.