Detecting Buggy Contracts via Smart Testing

TL;DR

SmartSys is an innovative system that enhances hybrid dynamic analysis of smart contracts by teaching foundation models to identify performance bottlenecks and generate effective fuzz targets, leading to improved bug detection and deeper coverage.

Contribution

We introduce SmartSys, a self-deciding foundation model system that improves hybrid smart contract testing by forecasting analysis techniques and reducing hallucinations.

Findings

Discovered a long-escaped smart contract vulnerability.

Achieved up to 14.3% coverage improvement on benchmarks.

Detected a vulnerability missed by eleven tools and multiple audits.

Abstract

Smart contracts are susceptible to critical vulnerabilities. Hybrid dynamic analyses, such as concolic execution assisted fuzzing and foundation model assisted fuzzing, have emerged as highly effective testing techniques for smart contract bug detection recently. This hybrid approach has shown initial promise in real-world benchmarks, but it still suffers from low scalability to find deep bugs buried in complex code patterns. We observe that performance bottlenecks of existing dynamic analyses and model hallucination are two main factors limiting the scalability of this hybrid approach in finding deep bugs. To overcome the challenges, we design an interactive, self-deciding foundation model based system, called SmartSys, to support hybrid smart contract dynamic analyses. The key idea is to teach foundation models about performance bottlenecks of different dynamic analysis techniques, making it possible to forecast the right technique and generates effective fuzz targets that can reach deep, hidden bugs. To prune hallucinated, incorrect fuzz targets, SmartSys feeds foundation models with feedback from dynamic analysis during compilation and at runtime. The interesting results of SmartSys include: i) discovering a smart contract protocol vulnerability that has escaped eleven tools and survived multiple audits for over a year; ii) improving coverage by up to 14.3\% on real-world benchmarks compared to the baselines.