Detecting Various DeFi Price Manipulations with LLM Reasoning

TL;DR

DeFiScope is a novel LLM-based system that effectively detects a wide range of DeFi price manipulation attacks, including non-standard models, outperforming existing methods in accuracy and real-world validation.

Contribution

This paper introduces DeFiScope, the first LLM-based approach for detecting diverse DeFi price manipulations, leveraging synthesized on-chain data and high-level transaction analysis.

Findings

Achieves 80% recall and 96% precision on real-world attacks.

Detects 147 real-world manipulation incidents, including 81 unknown cases.

Outperforms state-of-the-art methods significantly.

Abstract

DeFi (Decentralized Finance) is one of the most important applications of today's cryptocurrencies and smart contracts. It manages hundreds of billions in Total Value Locked (TVL) on-chain, yet it remains susceptible to common DeFi price manipulation attacks. Despite state-of-the-art (SOTA) systems like DeFiRanger and DeFort, we found that they are less effective to non-standard price models in custom DeFi protocols, which account for 44.2% of the 95 DeFi price manipulation attacks reported over the past three years. In this paper, we introduce the first LLM-based approach, DeFiScope, for detecting DeFi price manipulation attacks in both standard and custom price models. Our insight is that large language models (LLMs) have certain intelligence to abstract price calculation from smart contract source code and infer the trend of token price changes based on the extracted price models. To further strengthen LLMs in this aspect, we leverage Foundry to synthesize on-chain data and use it to fine-tune a DeFi price-specific LLM. Together with the high-level DeFi operations recovered from low-level transaction data, DeFiScope detects various DeFi price manipulations according to systematically mined patterns. Experimental results show that DeFiScope achieves a high recall of 80% on real-world attacks, a precision of 96% on suspicious transactions, and zero false alarms on benign transactions, significantly outperforming SOTA approaches. Moreover, we evaluate DeFiScope's cost-effectiveness and demonstrate its practicality by helping our industry partner confirm 147 real-world price manipulation attacks, including discovering 81 previously unknown historical incidents.