A Systematic Study on Static Control Flow Obfuscation Techniques in Java

TL;DR

This paper systematically analyzes static control flow obfuscation techniques in Java, revealing gaps between research and commercial tools, and proposes a new classification to improve understanding and development of effective obfuscation methods.

Contribution

It provides the first comprehensive classification of CFO techniques, compares literature and tools, and identifies gaps in implementation against potent obfuscation methods.

Findings

36 unique techniques in literature, 7 in tools

Most tools implement only 13 of the 36 techniques

Majority of tools lack potent obfuscation techniques

Abstract

Control flow obfuscation (CFO) alters the control flow path of a program without altering its semantics. Existing literature has proposed several techniques; however, a quick survey reveals a lack of clarity in the types of techniques proposed, and how many are unique. What is also unclear is whether there is a disparity in the theory and practice of CFO. In this paper, we systematically study CFO techniques proposed for Java programs, both from papers and commercially available tools. We evaluate 13 obfuscators using a dataset of 16 programs with varying software characteristics, and different obfuscator parameters. Each program is carefully reverse engineered to study the effect of obfuscation. Our study reveals that there are 36 unique techniques proposed in the literature and 7 from tools. Three of the most popular commercial obfuscators implement only 13 of the 36 techniques in the literature. Thus there appears to be a gap between the theory and practice of CFO. We propose a novel classification of the obfuscation techniques based on the underlying component of a program that is transformed. We identify the techniques that are potent against reverse engineering attacks, both from the perspective of a human analyst and an automated program decompiler. Our analysis reveals that majority of the tools do not implement these techniques, thus defeating the protection obfuscation offers. We furnish examples of select techniques and discuss our findings. To the best of our knowledge, we are the first to assemble such a research. This study will be useful to software designers to decide upon the best techniques to use based upon their needs, for researchers to understand the state-of-the-art and for commercial obfuscator developers to develop new techniques.