Automated Test-Case Generation for Solidity Smart Contracts: the AGSolT Approach and its Evaluation

TL;DR

This paper introduces AGSOLT, an automated tool for generating test suites for Solidity smart contracts, demonstrating high coverage and error detection capabilities through genetic and random search algorithms.

Contribution

The paper presents AGSOLT, a novel automated test suite generator for Solidity contracts that effectively uses search algorithms to improve testing coverage and find errors.

Findings

Achieves high branch coverage on real-world contracts

Discovered errors in popular Solidity contracts

Genetic algorithm outperforms random search in some cases

Abstract

Blockchain and smart contract technology are novel approaches to data and code management that facilitate trusted computing by allowing for development in a distributed and decentralized manner. Testing smart contracts comes with its own set of challenges which have not yet been fully identified and explored. Although existing tools can identify and discover known vulnerabilities and their interactions on the Ethereum blockchain through random search or symbolic execution, these tools generally do not produce test suites suitable for human oracles. In this paper, we present AGSOLT (Automated Generator of Solidity Test Suites). We demonstrate its efficiency by implementing two search algorithms to automatically generate test suites for stand-alone Solidity smart contracts, taking into account some of the blockchain-specific challenges. To test AGSOLT, we compared a random search algorithm and a genetic algorithm on a set of 36 real-world smart contracts. We found that AGSOLT is capable of achieving high branch coverage with both approaches and even discovered some errors in some of the most popular Solidity smart contracts on Github.