SuMo: A Mutation Testing Strategy for Solidity Smart Contracts

TL;DR

SuMo is a mutation testing tool for Ethereum smart contracts that uses 44 specialized operators to evaluate test suite effectiveness and improve contract reliability.

Contribution

The paper introduces SuMo, a novel mutation testing tool with a comprehensive set of operators tailored for Solidity smart contracts, enhancing fault detection.

Findings

SuMo effectively identifies weaknesses in test suites.

The tool reduces generation of stillborn mutants, improving efficiency.

Initial evaluations show promising results in increasing contract reliability.

Abstract

Smart Contracts are software programs that are deployed and executed within a blockchain infrastructure. Due to their immutable nature, directly resulting from the specific characteristics of the deploying infrastructure, smart contracts must be thoroughly tested before their release. Testing is one of the main activities that can help to improve the reliability of a smart contract, so as to possibly prevent considerable loss of valuable assets. It is therefore important to provide the testers with tools that permit them to assess the activity they performed. Mutation testing is a powerful approach for assessing the fault-detection capability of a test suite. In this paper, we propose SuMo, a novel mutation testing tool for Ethereum Smart Contracts. SuMo implements a set of 44 mutation operators that were designed starting from the latest Solidity documentation, and from well-known mutation testing tools. These allow to simulate a wide variety of faults that can be made by smart contract developers. The set of operators was designed to limit the generation of stillborn mutants, which slow down the mutation testing process and limit the usability of the tool. We report a first evaluation of SuMo on open-source projects for which test suites were available. The results we got are encouraging, and they suggest that SuMo can effectively help developers to deliver more reliable smart contracts.