Multi-View Pre-Trained Model for Code Vulnerability Identification

TL;DR

This paper introduces MV-PTM, a multi-view pre-trained model that encodes sequential and structural code information, significantly improving vulnerability detection accuracy over existing models.

Contribution

The paper presents a novel multi-view pre-trained model that incorporates multiple structural views of code using contrastive learning, enhancing vulnerability identification.

Findings

MV-PTM outperforms GraphCodeBERT by 3.36% F1 score on average.

Encoding multiple structural views improves vulnerability detection.

Contrastive learning enhances code representation quality.

Abstract

Vulnerability identification is crucial for cyber security in the software-related industry. Early identification methods require significant manual efforts in crafting features or annotating vulnerable code. Although the recent pre-trained models alleviate this issue, they overlook the multiple rich structural information contained in the code itself. In this paper, we propose a novel Multi-View Pre-Trained Model (MV-PTM) that encodes both sequential and multi-type structural information of the source code and uses contrastive learning to enhance code representations. The experiments conducted on two public datasets demonstrate the superiority of MV-PTM. In particular, MV-PTM improves GraphCodeBERT by 3.36\% on average in terms of F1 score.