Mutation Testing for Ethereum Smart Contract

TL;DR

This paper introduces a mutation testing framework for Ethereum smart contracts, featuring 15 novel mutation operators, demonstrating high defect detection effectiveness and revealing real bugs in Ethereum DApps.

Contribution

It presents the first systematic mutation testing approach for Ethereum smart contracts with 15 new mutation operators and empirical validation on real-world DApps.

Findings

96.01% defect detection rate with mutation testing

117 real bugs related to proposed mutation operators

Outperforms coverage-based approach in defect detection

Abstract

Smart contract is a special program that manages digital assets on blockchain. It is difficult to recover the loss if users make transactions through buggy smart contracts, which cannot be directly fixed. Hence, it is important to ensure the correctness of smart contracts before deploying them. This paper proposes a systematic framework to mutation testing for smart contracts on Ethereum, which is currently the most popular open blockchain for deploying and running smart contracts. Fifteen novel mutation operators have been designed for Ethereum Smart Contracts (ESC), in terms of keyword, global variable/function, variable unit, and error handling. An empirical study on 26 smart contracts in four Ethereum DApps has been conducted to evaluate the effectiveness of mutation testing. The experimental results show that our approach can outperform the coverage-based approach on defect detection rate (96.01% vs. 55.68%). The ESC mutation operators are effective to reveal real defects and we found 117 out of 729 real bug reports are related to our operators. These show the great potential of using mutation testing for quality assurance of ESC.