White-Basilisk: A Hybrid Model for Code Vulnerability Detection

TL;DR

White-Basilisk is a compact, innovative AI model that outperforms larger models in detecting code vulnerabilities, capable of analyzing extensive codebases efficiently and effectively.

Contribution

The paper introduces White-Basilisk, a novel hybrid architecture achieving state-of-the-art vulnerability detection with only 200M parameters and processing long code sequences.

Findings

White-Basilisk surpasses existing models in vulnerability detection accuracy.

It handles longer code sequences than current LLMs.

The model maintains high performance on real-world, imbalanced datasets.

Abstract

The proliferation of software vulnerabilities presents a significant challenge to cybersecurity, necessitating more effective detection methodologies. We introduce White-Basilisk, a novel approach to vulnerability detection that demonstrates superior performance while challenging prevailing assumptions in AI model scaling. Utilizing an innovative architecture that integrates Mamba layers, linear self-attention, and a Mixture of Experts framework, White-Basilisk achieves state-of-the-art results in vulnerability detection tasks with a parameter count of only 200M. The model's capacity to process sequences of unprecedented length enables comprehensive analysis of extensive codebases in a single pass, surpassing the context limitations of current Large Language Models (LLMs). White-Basilisk exhibits robust performance on imbalanced, real-world datasets, while maintaining computational efficiency that facilitates deployment across diverse organizational scales. This research not only establishes new benchmarks in code security but also provides empirical evidence that compact, efficiently designed models can outperform larger counterparts in specialized tasks, potentially redefining optimization strategies in AI development for domain-specific applications.