Efficient Software Vulnerability Detection Using Transformer-based Models

TL;DR

This paper introduces a transformer-based approach for C/C++ vulnerability detection that captures both local and global code context, improving accuracy and efficiency over previous neural network methods.

Contribution

It applies transformer models to vulnerability detection using program slices that encode key syntactic and semantic features, enhancing global context understanding.

Findings

Transformer models improve vulnerability detection accuracy.

The approach balances resource use and training time effectively.

Program slices effectively encode key code features.

Abstract

Detecting software vulnerabilities is critical to ensuring the security and reliability of modern computer systems. Deep neural networks have shown promising results on vulnerability detection, but they lack the capability to capture global contextual information on vulnerable code. To address this limitation, we explore the application of transformers for C/C++ vulnerability detection. We use program slices that encapsulate key syntactic and semantic features of program code, such as API function calls, array usage, pointer manipulations, and arithmetic expressions. By leveraging transformers' capability to capture both local and global contextual information on vulnerable code, our work can identify vulnerabilities accurately. Combined with data balancing and hyperparameter fine-tuning, our work offers a robust and efficient approach to identifying vulnerable code with moderate resource usage and training time.