One-for-All Does Not Work! Enhancing Vulnerability Detection by Mixture-of-Experts (MoE)

TL;DR

This paper introduces MoEVD, a Mixture-of-Experts framework for vulnerability detection that improves accuracy across various CWE types, especially underrepresented ones, by decomposing the task into specialized experts.

Contribution

The paper proposes MoEVD, a novel Mixture-of-Experts approach that decomposes vulnerability detection into classification and specific detection tasks, enhancing performance over existing models.

Findings

MoEVD achieves an F1-score of 0.44, outperforming SOTA baselines by at least 12.8%.

MoEVD improves recall on almost all CWE types, with gains from 9% to 77.8%.

MoEVD enhances performance on long-tailed CWE types by at least 7.3%.

Abstract

Deep Learning-based Vulnerability Detection (DLVD) techniques have garnered significant interest due to their ability to automatically learn vulnerability patterns from previously compromised code. Despite the notable accuracy demonstrated by pioneering tools, the broader application of DLVD methods in real-world scenarios is hindered by significant challenges. A primary issue is the "one-for-all" design, where a single model is trained to handle all types of vulnerabilities. This approach fails to capture the patterns of different vulnerability types, resulting in suboptimal performance, particularly for less common vulnerabilities that are often underrepresented in training datasets. To address these challenges, we propose MoEVD, which adopts the Mixture-of-Experts (MoE) framework for vulnerability detection. MoEVD decomposes vulnerability detection into two tasks, CWE type classification and CWE-specific vulnerability detection. By splitting the task, in vulnerability detection, MoEVD allows specific experts to handle distinct types of vulnerabilities instead of handling all vulnerabilities within one model. Our results show that MoEVD achieves an F1-score of 0.44, significantly outperforming all studied state-of-the-art (SOTA) baselines by at least 12.8%. MoEVD excels across almost all CWE types, improving recall over the best SOTA baseline by 9% to 77.8%. Notably, MoEVD does not sacrifice performance on long-tailed CWE types; instead, its MoE design enhances performance (F1-score) on these by at least 7.3%, addressing long-tailed issues effectively.