PrAIoritize: Automated Early Prediction and Prioritization of Vulnerabilities in Smart Contracts

TL;DR

PrAIoritize is an automated tool that uses advanced NLP and machine learning techniques to predict and prioritize vulnerabilities in Ethereum smart contracts during code review, improving efficiency and accuracy.

Contribution

This paper introduces PrAIoritize, a novel automated approach combining LLMs and NLP for early vulnerability prediction and prioritization in smart contract code reviews.

Findings

Significant improvement over state-of-the-art baselines in classification metrics

Achieved 4.82%-27.94% increase in F-measure, precision, and recall

Effectively reduces manual effort in smart contract vulnerability triage

Abstract

Context:Smart contracts are prone to numerous security threats due to undisclosed vulnerabilities and code weaknesses. In Ethereum smart contracts, the challenges of timely addressing these code weaknesses highlight the critical need for automated early prediction and prioritization during the code review process. Efficient prioritization is crucial for smart contract security. Objective:Toward this end, our research aims to provide an automated approach, PrAIoritize, for prioritizing and predicting critical code weaknesses in Ethereum smart contracts during the code review process. Method: To do so, we collected smart contract code reviews sourced from Open Source Software (OSS) on GitHub and the Common Vulnerabilities and Exposures (CVE) database. Subsequently, we developed PrAIoritize, an innovative automated prioritization approach. PrAIoritize integrates advanced Large Language Models (LLMs) with sophisticated natural language processing (NLP) techniques. PrAIoritize automates code review labeling by employing a domain-specific lexicon of smart contract weaknesses and their impacts. Following this, feature engineering is conducted for code reviews, and a pre-trained DistilBERT model is utilized for priority classification. Finally, the model is trained and evaluated using code reviews of smart contracts. Results: Our evaluation demonstrates significant improvement over state-of-the-art baselines and commonly used pre-trained models (e.g. T5) for similar classification tasks, with 4.82\%-27.94\% increase in F-measure, precision, and recall. Conclusion: By leveraging PrAIoritize, practitioners can efficiently prioritize smart contract code weaknesses, addressing critical code weaknesses promptly and reducing the time and effort required for manual triage.