SmartBugs: A Framework to Analyze Solidity Smart Contracts

TL;DR

SmartBugs is an extensible framework that streamlines the execution and comparison of analysis tools on Solidity smart contracts, facilitating research reproducibility and large-scale vulnerability detection.

Contribution

It introduces a flexible framework supporting multiple analysis tools and datasets, enabling comprehensive evaluation and comparison of smart contract analysis methods.

Findings

Supported the largest experimental setup to date in this domain.

Enhanced SmartCheck to improve detection of specific vulnerabilities, increasing identified vulnerabilities from 11% to 24%.

Provided datasets for evaluating analysis tool precision and large-scale contract collection.

Abstract

Over the last few years, there has been substantial research on automated analysis, testing, and debugging of Ethereum smart contracts. However, it is not trivial to compare and reproduce that research. To address this, we present SmartBugs, an extensible and easy-to-use execution framework that simplifies the execution of analysis tools on smart contracts written in Solidity, the primary language used in Ethereum. SmartBugs is currently distributed with support for 10 tools and two datasets of Solidity contracts. The first dataset can be used to evaluate the precision of analysis tools, as it contains 143 annotated vulnerable contracts with 208 tagged vulnerabilities. The second dataset contains 47,518 unique contracts collected through Etherscan. We discuss how SmartBugs supported the largest experimental setup to date both in the number of tools and in execution time. Moreover, we show how it enables easy integration and comparison of analysis tools by presenting a new extension to the tool SmartCheck that improves substantially the detection of vulnerabilities related to the DASP10 categories Bad Randomness, Time Manipulation, and Access Control (identified vulnerabilities increased from 11% to 24%).