Line-level Semantic Structure Learning for Code Vulnerability Detection

TL;DR

This paper introduces CSLS, a novel code vulnerability detection model that leverages line-level semantic and structural information, outperforming existing methods by capturing code's inherent structural nuances.

Contribution

The paper proposes a line-level semantic learning approach that retains structural elements and models nonlinear relationships, enhancing vulnerability detection accuracy.

Findings

Achieved 70.57% accuracy on Devign dataset.

Attained 49.59% F1 score on Reveal dataset.

Outperformed state-of-the-art baselines in vulnerability detection.

Abstract

Unlike the flow structure of natural languages, programming languages have an inherent rigidity in structure and grammar.However, existing detection methods based on pre-trained models typically treat code as a natural language sequence, ignoring its unique structural information. This hinders the models from understanding the code's semantic and structual information.To address this problem, we introduce the Code Structure-Aware Network through Line-level Semantic Learning (CSLS), which comprises four components: code preprocessing, global semantic awareness, line semantic awareness, and line semantic structure awareness.The preprocessing step transforms the code into two types of text: global code text and line-level code text.Unlike typical preprocessing methods, CSLS retains structural elements such as newlines and indent characters to enhance the model's perception of code lines during global semantic awareness.For line semantics structure awareness, the CSLS network emphasizes capturing structural relationships between line semantics.Different from the structural modeling methods based on code blocks (control flow graphs) or tokens, CSLS uses line semantics as the minimum structural unit to learn nonlinear structural relationships, thereby improving the accuracy of code vulnerability detection.We conducted extensive experiments on vulnerability detection datasets from real projects. The CSLS model outperforms the state-of-the-art baselines in code vulnerability detection, achieving 70.57% accuracy on the Devign dataset and a 49.59% F1 score on the Reveal dataset.