Agentic Discovery and Validation of Android App Vulnerabilities

TL;DR

A2 is a system that improves Android vulnerability detection by combining semantic analysis with validation techniques, reducing false positives and providing validated exploitability evidence, thus enhancing security assessment efficiency.

Contribution

A2 introduces a novel two-phase approach that mirrors expert analysis, combining semantic understanding with multi-modal validation to improve vulnerability detection accuracy.

Findings

Achieves 78.3% coverage on Ghera benchmark, surpassing state-of-the-art analyzers.

Distills results into 82 speculative findings with 47 Ghera cases and 28 true positives.

Uncovers 104 true-positive zero-day vulnerabilities in real-world APKs, with over half validated by PoCs.

Abstract

Existing Android vulnerability detection tools overwhelm teams with thousands of low-signal warnings yet uncover few true positives. Analysts spend days triaging these results, creating a bottleneck in the security pipeline. Meanwhile, genuinely exploitable vulnerabilities often slip through, leaving opportunities open to malicious counterparts. We introduce A2, a system that mirrors how security experts analyze and validate Android vulnerabilities through two complementary phases: (i) Agentic Vulnerability Discovery, which reasons about application security by combining semantic understanding with traditional security tools; and (ii) Agentic Vulnerability Validation, which systematically validates vulnerabilities across Android's multi-modal attack surface-UI interactions, inter-component communication, file system operations, and cryptographic computations. On the Ghera benchmark (n=60), A2 achieves 78.3% coverage, surpassing state-of-the-art analyzers (e.g., APKHunt 30.0%). Rather than overwhelming analysts with thousands of warnings, A2 distills results into 82 speculative vulnerability findings, including 47 Ghera cases and 28 additional true positives. Crucially, A2 then generates working Proof-of-Concepts (PoCs) for 51 of these speculative findings, transforming them into validated vulnerability findings that provide direct, self-confirming evidence of exploitability. In real-world evaluation on 169 production APKs, A2 uncovers 104 true-positive zero-day vulnerabilities. Among these, 57 (54.8%) are self-validated with automatically generated PoCs, including a medium-severity vulnerability in a widely used application with over 10 million installs.

Tables

Table 1. TABLE 1: Examples of tools that both A2 and adversaries may use.

Capability Category	Example Supporting Tools
Code Analysis	Jadx [25]
Manifest Inspection	Androidguard [26]
Static Vulnerability Scanning	MobSF [5], APKHunt [6]
Dynamic Runtime Control	Android Emulator [27]
Large Language Models	OpenAI, Google Gemini, etc.

Table 2. TABLE 3: Comparison of static tools evaluated in this study

Tool	Stars	Commits	Version	Language
MobSF [5]	19k+	2,000+	v4.3.2	Python
APKHunt [6]	800+	100+	v1.0.0	Go
Trueseeing [35]	60+	850+	v2.2.7	Python

Table 3. TABLE 4: Summary of A2 detection results on the Ghera benchmark and real-world APKs. APK is the number of applications analyzed. LoC is the average lines of code per application. Alert is the number of vulnerabilities initially reported by A2 . Stage distinguishes Agentic Vulnerability Discovery and Agentic Vulnerability Validation. TP and FP are true and false positives. B_TP is the number of Ghera ground-truth vulnerabilities detected (out of 60). Validation consistently reduces false positives (e.g., Ghera: 8→1, real-world APKs: 32→3) while preserving most true positives.

Dataset	APK	LoC	Alert	Stage	TP	FP	B_TP
Ghera	60	46,100	82	Agentic Vulnerability Discovery	75	7	47
				Agentic Vulnerability Validation	51	1	34
Production	169	115,696	136	Agentic Vulnerability Discovery	104	32	–
				Agentic Vulnerability Validation	57	3	–

Table 4. TABLE 5: Classification of True Positive (TP) Test Cases by Vulnerability Type. Speculative refers to speculative vulnerability findings, and Validated refers to validated vulnerability findings.

Type	Speculative	Validated
Authorization	19	9
Network Communication	23	1
Code Injection	15	12
WebView	13	9
Data Exposure	22	17
Inter-Component Communication	12	9
Total	104	57

Table 5. TABLE 6: Vulnerability detection results from three SAST tools and four large language models, and aggregation results from four models on the Ghera dataset. Vulnerabilities are organized into eight categories: Crypto, ICC, Networking, NonAPI, Permission, Storage, System, and Web. Numbers indicate reported issue counts; checkmarks ( ✓ \checkmark ) or crosses ( × \times ) indicate successful or missed detections. Asterisks ( ∗ ) mark vulnerabilities detected only by LLM.

Vulnerability Name	Vulnerability Detection						Vulnerability Aggregation
	MobSF	APKHu.	Trues.	G2.5F	G2.5P	OSS	O3	G2.5F	G2.5P	OSS	O3
BlockCipher-ECB-InformationExposure	95 $✓$	25 $✓$	18 $✓$	3 $\times$	3 $✓$	6 $✓$	4 $✓$	3 $✓$	3 $✓$	4 $✓$	3 $✓$
BlockCipher-NonRandomIV-InformationExposure∗	91 $\times$	24 $\times$	18 $\times$	3 $✓$	2 $✓$	5 $✓$	4 $✓$	2 $✓$	2 $✓$	3 $✓$	3 $✓$
ConstantKey-ForgeryAttack	107 $\times$	22 $\times$	17 $✓$	2 $✓$	2 $✓$	4 $✓$	3 $✓$	1 $✓$	1 $✓$	3 $✓$	2 $✓$
ExposedCredentials-InformationExposure∗	91 $\times$	23 $\times$	16 $\times$	1 $✓$	1 $✓$	3 $✓$	2 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
PBE-ConstantSalt-InformationExposure∗	95 $\times$	24 $\times$	18 $\times$	3 $\times$	3 $✓$	4 $✓$	3 $✓$	3 $✓$	2 $✓$	2 $✓$	3 $✓$
DynamicRegBroadcastReceiver-UnrestrictedAccess∗	92 $\times$	23 $\times$	16 $\times$	1 $✓$	1 $✓$	2 $✓$	2 $✓$	2 $✓$	1 $✓$	3 $✓$	1 $✓$
EmptyPendingIntent-PrivEscalation∗	94 $\times$	22 $\times$	15 $\times$	1 $✓$	1 $✓$	3 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$
FragmentInjection-PrivEscalation∗	94 $\times$	21 $\times$	15 $\times$	1 $✓$	1 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$
HighPriority-ActivityHijack	94 $\times$	21 $✓$	15 $\times$	2 $✓$	3 $✓$	2 $✓$	2 $✓$	3 $✓$	2 $✓$	3 $✓$	2 $✓$
ImplicitPendingIntent-IntentHijack∗	95 $\times$	22 $\times$	15 $\times$	2 $\times$	2 $\times$	3 $\times$	3 $✓$	2 $✓$	2 $✓$	3 $✓$	2 $✓$
InadequatePathPermission-InformationExposure∗	94 $\times$	23 $\times$	16 $\times$	1 $\times$	1 $\times$	4 $\times$	2 $✓$	1 $✓$	1 $✓$	3 $✓$	1 $✓$
IncorrectHandlingImplicitIntent-UnauthorizedAccess	92 $\times$	21 $\times$	15 $✓$	1 $✓$	1 $✓$	2 $✓$	3 $✓$	3 $✓$	1 $✓$	2 $✓$	2 $✓$
NoValidityCheckOnBroadcastMsg-UnintendedInvocation	95 $\times$	23 $\times$	17 $✓$	1 $\times$	2 $\times$	3 $✓$	2 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
OrderedBroadcast-DataInjection	93 $\times$	24 $\times$	17 $\times$	1 $\times$	1 $\times$	4 $\times$	1 $\times$	2 $\times$	1 $\times$	2 $\times$	1 $\times$
StickyBroadcast-DataInjection∗	94 $\times$	23 $\times$	16 $\times$	1 $✓$	1 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
TaskAffinity-ActivityHijack	94 $\times$	23 $\times$	16 $\times$	2 $\times$	2 $\times$	6 $\times$	1 $\times$	2 $\times$	1 $\times$	2 $\times$	1 $\times$
TaskAffinity-LauncherActivity-PhishingAttack	94 $\times$	21 $\times$	15 $\times$	0 $\times$	0 $\times$	3 $\times$	0 $\times$	1 $\times$	1 $\times$	3 $\times$	1 $\times$
TaskAffinity-PhishingAttack	94 $\times$	23 $\times$	16 $\times$	2 $\times$	2 $\times$	2 $\times$	1 $\times$	1 $\times$	1 $\times$	2 $\times$	1 $\times$
TaskAffinityAndReparenting-PhishingAndDoSAttack	91 $\times$	21 $\times$	15 $\times$	1 $\times$	2 $\times$	7 $\times$	1 $\times$	2 $\times$	1 $\times$	4 $\times$	2 $\times$
UnhandledException-DOS∗	89 $\times$	20 $\times$	13 $\times$	1 $✓$	1 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
UnprotectedBroadcastRecv-PrivEscalation	91 $\times$	24 $✓$	18 $✓$	1 $✓$	2 $✓$	2 $✓$	1 $✓$	2 $✓$	2 $✓$	1 $✓$	1 $✓$
WeakChecksOnDynamicInvocation-DataInjection∗	91 $\times$	23 $\times$	17 $\times$	1 $✓$	1 $✓$	3 $✓$	2 $✓$	2 $✓$	1 $✓$	2 $✓$	1 $✓$
CheckValidity-InformationExposure	97 $✓$	25 $✓$	16 $\times$	2 $✓$	2 $✓$	3 $✓$	1 $✓$	1 $✓$	1 $✓$	3 $✓$	1 $✓$
IncorrectHostNameVerification-MITM	97 $✓$	25 $✓$	16 $\times$	2 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
InsecureSSLSocket-MITM	95 $\times$	23 $\times$	17 $\times$	1 $\times$	1 $\times$	1 $\times$	1 $\times$	1 $\times$	1 $\times$	1 $\times$	2 $\times$
InsecureSSLSocketFactory-MITM	95 $✓$	24 $\times$	16 $\times$	1 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$
InvalidCertificateAuthority-MITM	97 $✓$	25 $✓$	16 $\times$	3 $✓$	2 $✓$	4 $✓$	3 $✓$	1 $✓$	2 $✓$	2 $✓$	1 $✓$
OpenSocket-InformationLeak	91 $\times$	22 $\times$	17 $\times$	2 $\times$	1 $\times$	4 $\times$	1 $\times$	2 $\times$	2 $\times$	2 $\times$	2 $\times$
UnEncryptedSocketComm-MITM	97 $\times$	22 $\times$	17 $✓$	2 $✓$	1 $\times$	3 $✓$	1 $✓$	2 $✓$	1 $✓$	4 $✓$	2 $✓$
UnpinnedCertificates-MITM	96 $✓$	27 $✓$	17 $\times$	2 $✓$	3 $\times$	4 $\times$	1 $\times$	2 $✓$	2 $✓$	4 $✓$	2 $✓$
MergeManifest-UnintendedBehavior∗	94 $\times$	23 $\times$	16 $\times$	1 $✓$	2 $✓$	3 $✓$	2 $✓$	1 $✓$	2 $✓$	2 $✓$	2 $✓$
OutdatedLibrary-DirectoryTraversal∗	93 $\times$	21 $\times$	15 $\times$	1 $✓$	1 $✓$	5 $✓$	3 $✓$	2 $✓$	1 $✓$	3 $✓$	2 $✓$
UnnecessaryPerms-PrivEscalation	94 $\times$	23 $\times$	16 $\times$	1 $\times$	1 $\times$	3 $✓$	2 $\times$	2 $\times$	1 $\times$	2 $\times$	1 $\times$
WeakPermission-UnauthorizedAccess	91 $\times$	22 $✓$	15 $\times$	1 $\times$	1 $✓$	3 $\times$	2 $\times$	1 $✓$	1 $✓$	3 $✓$	1 $✓$
ExternalStorage-DataInjection	91 $\times$	21 $✓$	15 $\times$	1 $✓$	2 $✓$	1 $\times$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
ExternalStorage-InformationLeak	97 $\times$	21 $✓$	15 $\times$	1 $✓$	1 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
InternalStorage-DirectoryTraversal∗	94 $\times$	21 $\times$	15 $\times$	1 $✓$	1 $✓$	5 $✓$	2 $✓$	1 $✓$	1 $✓$	4 $✓$	1 $✓$
InternalToExternalStorage-InformationLeak∗	91 $\times$	22 $\times$	15 $\times$	1 $✓$	1 $✓$	1 $✓$	1 $\times$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
SQLite-execSQL	98 $\times$	24 $✓$	15 $\times$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	2 $✓$	1 $✓$	3 $✓$	2 $✓$
SQLite-RawQuery-SQLInjection	105 $\times$	25 $✓$	15 $\times$	1 $✓$	1 $✓$	3 $✓$	4 $✓$	3 $✓$	4 $✓$	4 $✓$	2 $✓$
SQLite-SQLInjection	101 $\times$	25 $✓$	15 $\times$	4 $✓$	1 $✓$	5 $✓$	4 $✓$	3 $✓$	4 $✓$	5 $✓$	3 $✓$
CheckCallingOrSelfPermission-PrivilegeEscalation∗	91 $\times$	23 $\times$	16 $\times$	1 $✓$	1 $\times$	4 $\times$	3 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
CheckPermission-PrivilegeEscalation	94 $\times$	23 $\times$	16 $\times$	2 $\times$	2 $\times$	3 $✓$	3 $\times$	1 $\times$	1 $\times$	4 $\times$	1 $\times$
ClipboardUse-InformationExposure	91 $\times$	23 $✓$	16 $\times$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	1 $✓$	3 $✓$	1 $✓$
DynamicCodeLoading-CodeInjection∗	94 $\times$	22 $\times$	16 $\times$	1 $✓$	1 $✓$	3 $✓$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
EnforceCallingOrSelfPermission-PrivilegeEscalation∗	91 $\times$	23 $\times$	16 $\times$	1 $\times$	1 $\times$	3 $✓$	3 $✓$	2 $✓$	1 $✓$	3 $✓$	2 $✓$
EnforcePermission-PrivilegeEscalation	91 $\times$	23 $\times$	16 $\times$	1 $\times$	1 $\times$	5 $\times$	2 $\times$	1 $\times$	2 $\times$	4 $\times$	1 $\times$
UniqueIDs-IdentityLeak	94 $\times$	22 $\times$	17 $✓$	1 $✓$	2 $✓$	2 $✓$	1 $✓$	2 $✓$	1 $✓$	3 $✓$	3 $✓$
HttpConnection-MITM	91 $\times$	23 $✓$	17 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$	2 $✓$	1 $✓$	1 $✓$	1 $✓$
JavaScriptExecution-CodeInjection	91 $\times$	25 $✓$	18 $\times$	2 $✓$	2 $✓$	3 $✓$	2 $✓$	2 $✓$	1 $✓$	2 $✓$	2 $✓$
UnsafeIntentURLImpl-InformationExposure∗	93 $\times$	25 $\times$	17 $\times$	1 $✓$	1 $✓$	5 $✓$	3 $✓$	2 $✓$	2 $✓$	3 $✓$	2 $✓$
WebView-CookieOverwrite	98 $\times$	24 $\times$	18 $\times$	1 $\times$	1 $\times$	3 $\times$	1 $\times$	1 $\times$	1 $\times$	2 $\times$	1 $\times$
WebView-NoUserPermission-InformationExposure	94 $✓$	25 $\times$	17 $\times$	2 $✓$	2 $✓$	3 $✓$	2 $✓$	1 $✓$	2 $✓$	3 $✓$	2 $✓$
WebViewAllowContentAccess-UnauthorizedFileAccess	94 $✓$	27 $\times$	18 $\times$	3 $✓$	2 $✓$	4 $✓$	2 $✓$	2 $✓$	1 $✓$	2 $✓$	2 $✓$
WebViewAllowFileAccess-UnauthorizedFileAccess	93 $✓$	26 $\times$	17 $\times$	1 $✓$	1 $✓$	4 $✓$	1 $✓$	1 $✓$	1 $✓$	3 $✓$	1 $✓$
WebViewIgnoreSSLWarning-MITM	95 $✓$	26 $✓$	18 $\times$	1 $✓$	1 $✓$	3 $✓$	1 $✓$	1 $✓$	1 $✓$	2 $✓$	1 $✓$
WebViewInterceptRequest-MITM	94 $\times$	30 $\times$	19 $\times$	3 $\times$	4 $\times$	5 $\times$	5 $\times$	2 $\times$	2 $\times$	3 $\times$	1 $\times$
WebViewLoadDataWithBaseUrl-UnauthorizedFileAccess	95 $✓$	25 $✓$	17 $\times$	1 $✓$	1 $✓$	4 $\times$	2 $✓$	1 $✓$	1 $✓$	5 $✓$	2 $✓$
WebViewOverrideUrl-MITM∗	94 $\times$	26 $\times$	18 $\times$	2 $✓$	2 $✓$	4 $✓$	3 $\times$	2 $✓$	2 $✓$	3 $✓$	2 $✓$
WebViewProceed-UnauthorizedAccess	102 $\times$	27 $\times$	18 $\times$	2 $\times$	3 $\times$	5 $\times$	3 $\times$	2 $\times$	1 $\times$	4 $\times$	2 $\times$
Reported Security Issus	5654	1405	978	89	92	193	116	95	82	157	91
Detected APK Vulnerabilities	11/60	18/60	8/60	40/60	40/60	42/60	43/60	47/60	47/60	47/60	47/60
Vulnerability Recall Rate	18.3%	30.0%	13.3%	66.7%	66.7%	70.0%	71.7%	78.3%	78.3%	78.3%	78.3%

Equations

$$E_i=\frac{R_i^{Ghera}\times V_{total}}{A_i}$$

Full text

We introduce A2, a system that mirrors how security experts analyze and validate Android vulnerabilities through two complementary phases: (i) A**gentic Vulnerability Discovery, which reasons about application security by combining semantic understanding with traditional security tools; and (ii) A**gentic Vulnerability Validation, which systematically validates vulnerabilities across Android’s multi-modal attack surface—UI interactions, inter-component communication, file system operations, and cryptographic computations.

On the Ghera benchmark ($n{=}60$), A2 achieves $78.3\%$ coverage, surpassing state-of-the-art analyzers (e.g., APKHunt $30.0\%$). Rather than overwhelming analysts with thousands of warnings, A2 distills results into 82 speculative vulnerability findings, including $47$ Ghera cases and $28$ additional true positives. Crucially, A2 then generates working Proof-of-Concepts for $51$ of these speculative findings, transforming them into validated vulnerability findings that provide direct, self-confirming evidence of exploitability.

In real-world evaluation on 169 production APKs, A2 uncovers 104 true-positive zero-day vulnerabilities. Among these, 57 ($54.8\%$) are self-validated with automatically generated PoCs, including a medium-severity vulnerability in a widely used application with over 10 million installs.

1 Introduction

Mobile applications are the primary interface to many digital services. The Google Play Store hosts approximately $2.8$ million Android applications [1]. These applications process sensitive user and enterprise data and implement business-critical workflows, making them attractive targets for adversaries. Reported mobile security incidents increased by $32\%$ year over year [2]. The scale of Android applications, together with their privileged access to device resources and user data, complicate comprehensive security assessment.

Prior work on APK vulnerability detection has largely relied on Static Application Security Testing (SAST) tools [3], such as FlowDroid [4], MobSF [5], and APKHunt [6]. While useful for surfacing known patterns, these tools leave material gaps in end-to-end security validation (cf. Figure 1).

• •

C1. Low Coverage: SAST tools match predefined patterns and data-flow rules. However, many Android vulnerabilities are context and execution dependent, and thus may evade static, rule-based detection systems.

• •

C2. High Volume of Warnings: SAST scans entire APKs, including dependencies, third-party libraries, and low-severity syntax issues, generating large volumes of low-signal warnings that overwhelm analysts and obscure issues in application code, driving costly manual triage [7, 8].

• •

C3. No Exploit Validation: SAST flags potential issues without writing PoC code to demonstrate and validate exploitability. Meanwhile, Dynamic Application Security Testing (DAST) tools [9] in theory can generate PoCs, but they struggle with Android’s structured, multimodal input space (UI events, Intents111In Android, an Intent is a message object that allows app components to request actions from each other. For example, an app can issue an implicit intent such as ACTION_VIEW with a URL to ask the system to open the link in a browser. While convenient, intents can also introduce security risks., filesystem, databases, shell), yielding a combinatorial search space that is challenging for fuzzers to explore effectively [10].

Recent progress in Large Language Models marks a fundamental shift in Android security analysis, overcoming long-standing limitations of traditional techniques. We present A2, a framework that mirrors the reasoning process of human analysts. At its core, A2 operates through two complementary phases: Agentic Vulnerability Discovery and Agentic Vulnerability Validation. The discovery phase reasons about application security, combining semantic code understanding with traditional security tools to form actionable vulnerability hypotheses. Building on these insights, the assessment phase systematically validates each hypothesis through coordinated agents that plan, execute, and verify exploitation attempts. Conceptually, A2 mirrors how security experts work: first understanding the application’s security posture comprehensively, then methodically proving or disproving each potential vulnerability through hands-on testing. The result is a system that not only identifies vulnerabilities but demonstrates their real-world impact through concrete, verifiable attack paths. Our primary contributions are:

• •

Human-Level End-to-end Security Analysis: To our knowledge, A2 is the first system to mimic expert practices in vulnerability assessment. It performs Agentic Vulnerability Discovery by combining semantic reasoning with traditional signals, and Agentic Vulnerability Validation by planning, executing, and verifying exploits. This human-like approach enables adaptive discovery beyond static rules while ensuring rigorous validation with concrete evidence.

• •

New State-of-the-Art (SOTA) on Ghera Benchmark: On 60 vulnerable APKs, we evaluate A2 alongside four reasoning LLMs and three SAST baselines. Among single models, o3 attains 71.7% (43/60). The ensemble of Gemini 2.5 Flash, Gemini 2.5 Pro, and o3 reaches 78.3% (47/60) and covers all $27$ vulnerabilities detected by the SAST tools. A2 aggregates $8,528$ outputs into $82$ speculative vulnerability findings while retaining all true positives. This shows that combining reasoning models with aggregation yields both higher coverage and significantly fewer speculative findings than existing approaches.

• •

LLM-based Vulnerability Validation: We evaluate A2 by generating PoC exploits for $82$ findings A2 reports on the Ghera benchmark. Within 20 iterations, using Gemini 2.5 Pro as the PoC Planner and Gemini 2.5 Flash as the Task Executor and Validator, A2 achieves a 61.3% success rate (46/75) on actionable vulnerabilities and correctly rules out all false positives (7/7). The Executor averages $4.49$ function calls per task, while the Validator filters 12.6% of erroneous claims through dynamically generated oracles, mitigating hallucination-induced false positives. When both the Executor and Validator are upgraded to Gemini 2.5 Pro, the success rate increases to 68.0% (+6.7%), and erroneous claims drop to 4.7% (-7.9%). Overall, these results demonstrate a shift from simple detection toward end-to-end validation of vulnerabilities through executable evidence.

• •

Zero Days: We deploy A2 on 169 AndroZoo [11] APKs released in 2024–2025, to assess A2’s effectiveness in practice. The vulnerability detection module reports 136 vulnerabilities, from which the agentic vulnerability validation module invalidates 29 false positives. Among the remaining 107 speculative vulnerability findings (104 true positives), A2 achieves a 54.8% success rate (57/104) in end-to-end exploitation using Gemini 2.5 Pro as the PoC Planner and Gemini 2.5 Flash as both Task Executor and Task Validator. We responsibly disclosed these zero-days, including cases in apps with over 10 million downloads.

2 Background

This section provides background on Android security.

Android Component Architecture

Android applications utilize a component-based architecture with four building blocks: Activities (UI screens), Services (background tasks), Broadcast Receivers (system events), and Content Providers (data access) [12]. These components communicate through Intents, which can be either explicit (targeting specific components) or implicit (system-resolved based on declared capabilities). The Android manifest file serves as the central configuration document, declaring all components along with their required permissions and export policies, effectively defining the application’s security boundaries. This modular design enables component reuse across applications, but simultaneously introduces security risks. Improperly configured exported components can be accessed by malicious applications, while malformed intents can lead to attacks such as intent spoofing and privilege escalation [13].

Android Application Vulnerability Landscape

The Android ecosystem faces a diverse landscape of security vulnerabilities that compromise user privacy and system integrity. A comprehensive empirical study by VulsTotal [14] systematically categorizes Android application vulnerabilities into 58 distinct types across six major categories: cryptographic weaknesses, inter-component communication flaws, networking vulnerabilities, permission misconfigurations, storage issues, and web-related problems. This taxonomy reveals the complexity of Android security, where vulnerabilities often arise from the interaction between multiple system components rather than isolated code defects. Traditional detection approaches focus primarily on well-known vulnerability patterns, but complex security flaws require deep semantic understanding of application logic and cross-component data flows that challenge conventional automated analysis techniques.

Static Application Security Testing Tools

SAST tools [3] are designed to statically analyze source code or compiled versions of code to identify security flaws. According to OWASP [15], static code analysis is performed as part of white-box testing during the implementation phase, using techniques such as taint analysis and data flow analysis to highlight possible vulnerabilities within non-running source code. These tools form the backbone of modern Android application vulnerability detection, providing automated analysis capabilities across diverse application codebases. Popular Android SAST tools include both shallow analysis tools such as MobSF [5], APKHunt [6], QARK [16], and AndroBugs [17], and deep taint analysis tools such as FlowDroid [4], Amandroid [18], and DroidSafe [19].

LLM-Powered Agents in Security

LLMs have shown strong code comprehension and analysis skills, leading to their growing use in software security [20]. Moving beyond rule-based tools, LLM-powered agents leverage contextual reasoning to perform multi-step security analysis with minimal supervision. Unlike static analysis tools that depend on fixed vulnerability patterns, they adapt to new vulnerability types by understanding semantic code relationships. Advances in prompt engineering, tool integration, and multi-agent systems have enabled tasks such as vulnerability classification, exploit generation, and automated penetration testing [21, 22]. Yet, LLM agents face limits in context windows, hallucination risks, and variable quality. Research addresses these through verification mechanisms and hybrid designs that combine LLM reasoning with traditional analysis [23].

Multi-Agent Orchestration

LangGraph [24] is a framework for organizing multi-agent workflows using graph structures. It represents agents as nodes and their interactions as edges, allowing developers to define how information and control flow between components. In practice, LangGraph acts as a programmable state machine: it coordinates message passing, supports looping and branching, and manages task execution order. While it is often presented as a new paradigm, its foundations align with long-standing concepts of workflow orchestration and state transition systems. Its main contribution is providing a convenient abstraction layer that makes it easier to connect LLMs with external tools, to structure multi-step reasoning pipelines, and to implement such systems in practice. In this work, A2 adopts LangGraph as it reduces our development overhead.

3 Models

This section defines the models and analysis scope.

System Model

We model the Android ecosystem through three actors: (i) developers, (ii) distribution channels, and (iii) end-user devices. Developers build applications that may contain vulnerabilities due to coding mistakes, misuse of APIs, or insecure third-party libraries. Applications are packaged as signed APKs, with identity tied to the signing key across updates. Source code is often closed, limiting visibility into developer practices. APKs are distributed through Google Play, third-party markets, and sideloading. For this study, we use AndroZoo [11], a growing dataset aggregating APKs from major markets, offering a representative sample of real-world apps. On devices, we assume each app executes in a sandbox defined by UID and SELinux, with permissions mediating access to system resources and co-resident apps. Apps interact with local components, embedded web content, and remote backend services, which define the trust boundaries in our model: per-app sandboxing, inter-app communication, web content, and network channels. We also assume stock, non-rooted Android devices with the platform security model enforced, and exclude physical access and custom ROMs.

Threat Model

We consider adversaries who attempt to exploit Android applications to steal sensitive data, escalate privileges across application boundaries, or tamper with application logic and communication channels. Such adversaries may start with only the APK, without access to source code or developer keys, but can analyze the package, execute it on a device or emulator, and craft exploit sequences that cross trust boundaries. We assume adversaries are capable of reverse engineering APKs, observing runtime behavior, and injecting crafted inputs through Android’s interaction channels, including Intents, UI events, filesystem operations, and network traffic. They do not control the Android platform, kernel, or hardware. Attacks requiring rooted devices, custom firmware, or hardware side channels are out of scope. Adversaries instead focus on application-layer vulnerabilities introduced by developers or insecure library use. Functional bugs [28, 29, 30] that do not compromise security properties are considered out of scope. To support these capabilities, adversaries may rely on existing tools (cf. Table 1), such as decompilers, manifest inspection utilities, static analysis suites, and emulators, which also form the basis for our AI agent (A2) when reasoning about attack strategies.

Terminology

To ensure precise language throughout this paper, we distinguish between three related terms:

Warning. A potential issue reported by static analysis or rule-based tools. Warnings may include false positives and are not supported by a PoC implementation. They require manual inspection to determine their relevance. 2. 2.

Speculative Vulnerability Finding. A candidate vulnerability generated by A2. Unlike warnings, these findings include a reasoned hypothesis about why the issue may be exploitable, but they remain unconfirmed until validation. 3. 3.

Validated Vulnerability Finding. A vulnerability for which at least one working PoC has been produced, and an LLM assessment indicates that the PoC demonstrates the vulnerability. While this stage goes beyond hypothesis, such findings may still include false positives if the PoC or validation is misleading or incomplete.

4 A2 Design

Figure 2 illustrates the end-to-end workflow of A2. Given an APK as input, A2 performs code analysis to generate speculative vulnerability findings. Through iterative PoC refinement and execution, A2 then transforms speculative vulnerability findings into validated vulnerability findings.

4.1 Agentic Vulnerability Discovery

A2 discovers vulnerabilities primarily through LLMs, with optional support from static tools and vulnerability databases.

4.1.1 LLM-based Vulnerability Analyzer

Our analyzer consists of three components (cf. Figure 2):

➀ APK Resource Extractor

(i) employs Jadx [25] to decompile bytecode into source code, then (ii) excludes third-party libraries and Android framework code, and (iii) extracts manifest details (e.g., component declarations, permission specifications, and inter-component communication policies.)

➁ LLM Vulnerability Analyzer

processes the code and manifest data using LLMs to identify security flaws and vulnerability patterns that may evade traditional rule-based detection. It generates standardized reports with exact line-level mappings in code, enabling the discovery of previously unseen vulnerabilities and context-dependent flaws that require understanding of application logic and data flow.

4.1.2 (Optional) Third-party Tool Integration

A2 can integrate third-party tools that accept an APK as input.

➂ Existing Static Tools

such as MobSF [5] and APKHunt [6] offer two advantages over pure LLMs: (i) they provide precise code locations and vulnerability context that enrich the evidence base, and (ii) their deterministic data flow analysis guides targeted exploitation by revealing concrete attack paths. This integration complements the semantic reasoning of LLMs with the precision of static analysis, enabling broader and more reliable vulnerability discovery.

➃ Security Warning Formatter

standardizes the diverse outputs of static analysis tools, which often vary in format and terminology. Leveraging LLMs, it harmonizes formats, aligns semantics, and structures reports into consistent security warnings for downstream processing.

4.1.3 Vulnerability Aggregation

The discovery process concludes with ➄, the Vulnerability Aggregator, an LLM-based component that unifies outputs from ➁ and ➃. The aggregator performs three key functions: (i) Security issue filtering, removing warnings unrelated to exploitable flaws such as code quality or performance issues; (ii) Semantic deduplication, collapsing duplicate findings across tools through similarity analysis and contextual reasoning; and (iii) Evidence synthesis, merging complementary signals to construct richer vulnerability profiles. Together, these functions yield speculative vulnerability findings, each representing a reasoned hypothesis about potential exploitable flaws, ready for validation in Section 4.2.

4.2 Agentic Vulnerability Validation

The Agentic Vulnerability Validation phase employs a multi-agent system to confirm speculative findings through PoC exploits. A2 uses LangGraph [24] for stateful coordination and task routing (Table 2). At its core, A2 runs a feedback loop: the PoC* Planner* breaks down a strategy into verifiable tasks, the Task Executor carries them out, and the Task Validator independently judges the results as success or failure.

➅ PoC Planner

performs strategic planning. For each speculative vulnerability finding, it examines the flaw’s characteristics, code context, and attack vectors to propose a step-by-step validation plan. As Table 2 shows, the ➅ Planner only triggers “read-only”222We use “read-only” to refer to function calls that do not alter the runtime state of the target application inside the Android emulator. However, they may still manipulate the local analysis environment, such as creating a python script, executing code in it, and collecting results. function calls, such as code navigation, file system analysis, and basic application control. This “read-only” design separates planning from execution and avoids unintended side effects. The ➅ Planner may also attempt to further filter false positives after analyzing all supporting evidence. After a false positive is speculated, the ➅ Planner assigns determine_potential_fp tasks to the ➆ Task Executor for concrete validation.

➆ Task Executor

implements the ➅ Planner’s natural-language strategy as a concrete PoC for speculative vulnerability findings. It sequentially executes the required steps using all eight toolkit categories ($29$ functions in total, cf. Table 2), covering code execution, device control, file system, static analysis, UI interaction, log analysis, APK generation, and web server management. Acting as a multimodal agent, the Executor processes both visual data (screenshots) and structural data (XML layouts) to navigate and interact with the victim application during PoC execution.

➇ Task Validator

provides independent verification of each PoC outcome. It never accepts the ➆ Task Executor’s self-reported success. Instead, after the ➆ Task Executor runs a PoC, the ➇ Task Validator uses its own observations and recomputation to judge whether the expected effect actually took place. ➇ Task Validator is the key to A2’s success.

Similar to ➅, the ➇ Task Validator operates with “read-only” function calls. This gives ➇ wide visibility but prevents any modification of the application state (which may change the PoC outcome, and we don’t want that, cf. Table 2).

Iterative Self-Correction

When execution fails at ➆ or validation rejects a success claim at ➇, feedback is sent back to the ➅ PoC Planner. The planner then inspects the failure, revises its strategy, and retries. This self-correction loop lets the system adapt and handle unexpected obstacles. The ➇ Task Validator guides the process into one of three states: (i) Continuation, when a task is validated and the next task begins; (ii) Re-planning, when validation fails and the planner must revise its approach; or (iii) Completion, when all tasks pass validation and the workflow ends.

4.3 Vulnerability Oracle

We highlight the oracle design as one of the core novelties of A2, since reliable validation is what ultimately separates speculative findings from actionable vulnerabilities.

Unlike some security domains where vulnerability oracles can be defined with clear heuristics—for example, in blockchain security, monetary gain can serve as a definitive indicator [31]—Android vulnerability validation presents a much harder challenge. Applications expose diverse behaviors, and strict rules that guarantee zero false positives are difficult, if not impossible, to design. This limitation has long constrained fuzzers and static analysis tools in the Android ecosystem: they either over-approximate and flag benign behaviors or under-approximate and miss subtle flaws.

A2 adopts a different strategy. Instead of relying on fixed heuristics, the ➇ Task Validator uses LLMs to generate customized oracles for each vulnerability, case by case. Each claim from the ➆ Executor is transformed into targeted verification questions. Guided by prior knowledge and the context of the current finding, the ➇ Validator designs oracles that independently test whether the claimed effect truly manifests. On a high level, an oracle involves the following steps:

Parse the claim. ➇ takes a single claim from the ➆ Executor, for example “hardcoded AES key found,” or “admin screen reachable without login.” 2. 2.

State the expected effect. ➇ writes the expected testable effect, for example “the key decrypts file correctly,” or “AdminActivity opens while the session is unauthenticated.” 3. 3.

Design an oracle. ➇ chooses one or more checks that can confirm or reject the effect. These checks use only “read only” tools from Table 2. 4. 4.

Collect evidence. ➇ run the selected tools to gather code, UI, logs, files, or network traces. 5. 5.

Decide. ➇ compare the evidence with the expected effect. The oracle returns PASS or FAIL, plus a short rationale and artifacts for the report.

We demonstrate A2’s validation capabilities through a hardcoded cryptographic key vulnerability case from the Ghera benchmark [32].

Vulnerability Scenario

The target application implements a password reset verification interface where users provide their email address and click "GET TOKEN" to receive a verification token. After entering the correct token, users proceed to the password reset interface. The application uses the Cipher API with AES encryption for token generation but stores the cryptographic key (0123456789!@#$%ˆ) directly in the strings.xml resource file. This enables attackers to extract the hardcoded key from the APK and forge authentication tokens for arbitrary users, bypassing security controls.

A2 validates this vulnerability through three sequential tasks mirroring a real-world attack scenario. Figures 3, 4, and 5 demonstrate the validation workflow, while Figure 6 shows the UI changes during exploitation.

Task 1: Hardcoded Key Extraction

In this initial validation step, A2 locates and extracts the hardcoded cryptographic key embedded within the application’s resource files. As shown in Figure 3, the system successfully identifies the AES key stored in the strings.xml configuration file. The Task Validator independently verifies both the file’s existence and the specific key value through oracle-based validation, confirming the presence of this critical security vulnerability.

Task 2: Password Reset Token Forgery

Building on the extracted key, A2 demonstrates the exploitability of this vulnerability by forging authentication tokens for arbitrary users. Figure 4 illustrates how the system uses the hardcoded key to encrypt a target email address (anniemaes@gmail.com) with AES encryption, generating a Base64-encoded password reset token. The validation process confirms that the generated token matches the expected cryptographic output and can successfully impersonate legitimate users.

Task 3: Authentication Bypass Validation

The final validation step proves that the forged tokens can bypass the application’s authentication mechanisms in practice. As demonstrated in Figure 5, A2 launches the password reset functionality using the crafted token, successfully deceiving the application into accepting it as legitimate. Figure 6 shows the complete UI progression across four states, documenting how the attack grants unauthorized access to password reset functionality, thereby confirming the end-to-end exploitability of the vulnerability.

5 Evaluation

In this section, we conduct experiments to evaluate A2’s performance, cost effectiveness, and capability to discover zero-day vulnerabilities in the wild. We compare A2 against SOTA static analysis tools and discss the quality of A2’s generated PoCs across multiple dimensions.

5.1 Dataset Selection

Our experiments are conducted on two datasets.

• •

The Ghera benchmark dataset contains 60 vulnerable Android APKs spanning eight categories—CRYPTO, ICC, NETWORKING, NON-API, PERMISSION, STORAGE, SYSTEM, and WEB—and serves as our primary testbed for both detection and exploit generation evaluation.

• •

For real-world vulnerability discovery, we use APKs from AndroZoo [11] released between January 2024 and July 2025. To address model context limits during detection, we restrict to APKs under $5$MB, yielding $169$ distinct samples: $126$ from play.google.com, $38$ from VirusShare, and $5$ from other platforms.

To reduce potential data leakage and model memorization, we remove all textual descriptions or semantic information about the vulnerabilities from our dataset. In addition, all application-specific identifiers, such as filenames and APK names, are replaced with generic placeholders.

5.2 Experimental Setup

We describe below the setup used for all experiments.

Computational Environment.

All experiments run on an Apple M4 system with 24GB RAM and 1TB SSD storage (macOS 26.0). The Android testing setup uses emulator version 35.6.11.0, adb 1.0.41 (build 36.0.0-13206524), and Android API level 36. Experiments are conducted with internet connectivity for LLM API access, with a 300-second timeout per call to ensure reliable execution.

Comparison Tool Selection.

We select tools based on three criteria: (i) APK compatibility, meaning the tool natively supports compiled Android Package files without requiring source code; (ii) broad vulnerability coverage, spanning multiple categories rather than focusing on a narrow class of flaws; and (iii) active maintenance and adoption, ensuring the tool is available and representative of current practice. These three selection criteria favor static tools. Dynamic analysis is particularly challenging for Android because of its multimodal input space – UI events, Intents, filesystem access, and database queries – which makes general-purpose DAST difficult to apply [10]. Existing Android DAST tools are usually specialized (e.g., fuzzers for media frameworks or native libraries [33, 34]) or UI fuzzers aimed at functional testing [28, 29, 30], rather than security vulnerabilities. In this paper, we evaluate three widely used SAST tools that cover a broad range of Android vulnerabilities: MobSF [5], APKHunt [6], and Trueseeing [35]. As of our paper submission, their features are summarized in Table 3.

LLM Selection.

To cover a representative range of performance and cost, we evaluate four commercial and open-source multimodal LLMs: OpenAI o3 (o3 2025-04-16), Gemini 2.5 Pro (gemini-2.5-pro), Gemini 2.5 Flash (gemini-2.5-flash), and GPT oss (gpt-oss-120b). At evaluation time, the advertised prices per million input and output tokens are 2 and 8 USD, 1.25 and 10 USD, 0.30 and 2.50 USD, and 0.10 and 0.50 USD, respectively. To keep reasoning capacity consistent across models, we set thinkingBudget to 24576 for gemini-2.5-flash and gemini-2.5-pro, and use the Medium thinking intensity for o3 and gpt-oss-120b.

5.3 Detection Performance

To ensure fairness, each LLM model is executed exactly once per APK. We collect all outputs directly, without cherry-picking or repeated attempts. For static tools, we use their default analysis pipelines. Ground-truth validation is performed manually by both authors, with cross-checking to confirm whether each tool correctly identifies the vulnerabilities documented in Ghera. Table 6 in the Appendix presents the full detection results.

Note that Ghera labels APKs with specific vulnerabilities but does not guarantee that other flaws are absent. For example, one APK labeled with BlockCipher-ECB-InformationExposure also contains a Hard-coded Cryptographic Key. Exhaustively identifying all vulnerabilities is non-trivial. To reduce subjective bias, we benchmark strictly against the vulnerabilities explicitly documented in Ghera.

Reported Findings Volume.

We observe a clear gap in the number of findings between SAST tools and our LLM-based analyzers. MobSF reports 5,654 findings, while o3 produces only 116. The large output of SAST tools comes from broad rules that flag application issues, third-party libraries, SDK usage, and even coding style. This adds noise and makes audits harder for analysts.

Detection Capability.

Figure 7 shows the number of vulnerabilities detected per tool, broken down by category. LLM-based analyzers achieve substantially higher recall than SAST tools. o3 detects 43 vulnerabilities (71.7%), while APKHunt detects 18 (30.0%) and MobSF only 11 (18.3%), despite MobSF’s much larger output volume. This contrast underscores the gap between concise LLM results and the noisy but shallow coverage of traditional static analysis.

LLM analyzers also achieve broader category coverage. They consistently detect more vulnerabilities in CRYPTO, ICC, and STORAGE, categories where SAST tools often fail. In comparison, SAST tools display narrow specialization: APKHunt and Trueseeing each identify unique cases, but their coverage is narrow. APKHunt detects four CRYPTO vulnerabilities that no other SAST tool reports, while Trueseeing is the only tool to flag three NONAPI issues and a single WEB case. Outside these isolated hits, their contribution is minimal, highlighting a lack of generalization beyond their targeted rules. Variance in performance further highlights this divide. SAST tools diverge widely because their rules are tied to specific detection oracles, while LLM analyzers maintain a more balanced profile across categories. Analysis shows that LLMs subsume and extend traditional detection. The three LLMs—gemini 2.5 flash, gemini 2.5 pro, and o3—collectively capture all 27 vulnerabilities found by the three SAST tools, while also uncovering 20 additional vulnerabilities that no SAST tool detects (marked with ∗ in Table 6). This demonstrates that LLM-based analysis not only preserves static-analysis strengths but also extends detection into previously unreachable ground. Despite these gains, 13 vulnerabilities remain undetected by any tool, a limitation discussed in Section 8.

5.4 Aggregation Effectiveness

The aggregation process collects and normalizes the heterogeneous outputs from all seven detection tools (three SAST tools and four LLM-based analyzers) into a unified StandardizedVulnerability format. For each of the 60 vulnerabilities in the Ghera benchmark, every aggregation model receives the complete set of outputs from all tools as input, regardless of whether individual tools flagged the vulnerability. We evaluate four LLM models as aggregation engines: gemini-2.5-flash (G2.5F), gemini-2.5-pro (G2.5P), o3, and gpt-oss-120b (oss). We record the outputs from each model and perform manual verification to check whether the aggregated results match the ground-truth vulnerabilities.

Coverage Analysis.

Table 6 in the Appendix shows that aggregation improves detection coverage. By combining the outputs from all seven tools, each of the four aggregation models identifies 47 of the 60 ground-truth vulnerabilities, achieving 78.3% recall. This is a clear gain over the best individual tool, o3, at 71.7%. The results highlight the complementary strengths of different detection approaches.

Consolidation Effectiveness.

While all four aggregation models achieve identical top-line recall performance, they exhibit notable differences in their consolidation strategies. For example, in analyzing SQLite-RawQuery-SQLInjection vulnerabilities, gemini-2.5-flash and gemini-2.5-pro consolidate findings into 4 distinct issues, whereas o3 reports only 2. Conversely, for ConstantKey-ForgeryAttack, o3 provides a more granular report with 2 issues compared to 1 from the other models. Notably, gpt-oss-120b demonstrates suboptimal consolidation performance, suggesting a higher tendency toward hallucination and less effective noise filtering compared to other models. Overall, gemini-2.5-flash, gemini-2.5-pro, and o3 demonstrate strong performance in producing concise, actionable final reports.

Detection Efficiency Analysis.

Security practitioners must balance detection coverage against analysis overhead. Traditional tools often produce thousands of noisy findings, complicating assessment and inflating audit costs. To capture this trade-off, we define a detection efficiency metric that relates recall to warning volume. Formally, the efficiency of tool $i$ is:

[TABLE]

where $R_{i}^{Ghera}$ is the recall of tool $i$ on the labeled vulnerabilities in the Ghera benchmark, $A_{i}$ is the number of warnings / speculative vulnerability findings it reports, and $V_{total}$ represents the true (but unknown) total number of vulnerabilities across the dataset. We only know $V_{total}\geq 60$, since 60 vulnerabilities are labeled.

Because $V_{total}$ is constant across all tools, relative comparisons depend only on the ratio $R_{i}^{Ghera}/A_{i}$. Intuitively, this ratio reflects the expected number of true findings per warning, scaled by recall. We use this proxy because it emphasizes the trade-off practitioners face: higher recall increases the chance of covering real vulnerabilities, while excessive warnings increase audit cost. Although other formulations (e.g., precision or F1-score) are possible, this measure directly captures the efficiency of converting warnings into actionable detections.

Figure 8 compares efficiency scores across tools. Three trends stand out. LLM advantage: LLM analyzers deliver higher recall (66.7–71.7%) while producing far fewer warnings (89–193) than SAST tools (978–5,654). Aggregation effectiveness: Combining outputs achieves the highest efficiency, reaching 78.3% recall with only 82–157 speculative vulnerability findings, highlighting the complementary strengths of different methods. SAST limitations: Static tools remain inefficient. MobSF, for example, generates over 5,000 warnings yet achieves only 18.3% recall.

5.5 Validation Effectiveness

We validate the 82 speculative vulnerability findings produced by gemini-2.5-pro in the aggregation stage. For clarity, we exclude two groups of findings from evaluation: (i) two findings from APKs that could not be installed on the emulator (WebViewAllowContentAccess-UnauthorizedFileAccess and WebViewLoadDataWithBaseUrl-UnauthorizedFileAccess); and (ii) seventeen findings corresponding to vulnerabilities outside A2’s current validation scope (see Section 8).

The remaining 63 (82-2-17) speculative vulnerability findings, listed in Table LABEL:tab:poc_summary in the Appendix, undergo manual review and classification. This process identifies 56 true positives and 7 false positives (marked in yellow in the table). To ensure accuracy, verification is cross-checked by multiple security researchers. We then evaluate these 63 findings under two model configurations for the three-agent system. The first configuration uses (gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash) for the (planner, executor, validator) roles, representing a cost-optimized setup. The second configuration uses (gemini-2.5-pro, gemini-2.5-pro, gemini-2.5-pro) across all roles, representing a performance-optimized setup with consistent model strength throughout the pipeline. For simplicity, we use Mixed Configuration and Unified Configuration to refer to these two setups respectively in subsequent sections. Validation outcomes are categorized with standardized symbols: ★ = true positive validated with a complete proof-of-concept; ✫ = false positive correctly identified and filtered; $\otimes$ = true positive misclassified as false positive; $\odot$ = false positive misclassified as true positive; $\times$ = execution terminated due to technical errors; $\bullet$ = validation stopped after reaching the maximum iteration limit.

Task Executor Performance.

Function calling success rates reach 92.5% (927/1002) for mixed configuration and 95.4% (649/680) for unified configuration, with the majority of failures concentrated in APK generation and web server management functions. These failures primarily result from environment-specific challenges including Gradle build system configuration conflicts, network port allocation issues, and Android emulator state management complexities. Figure 9 shows the function calls per task distribution.

The function calls per task distribution indicates that 80% of tasks complete within 2-4 function calls, with unified configuration demonstrating reduced function call requirements (average 2.92 vs 4.49 calls per task). This represents a +35% reduction in API overhead.

Validator Performance.

The Task Validator processes 223 task execution claims from the Task Executor in mixed configuration and 233 claims in unified configuration, achieving validation pass rates of 87.4% (195/223) and 95.3% (222/233) respectively. The validator identifies and filters 28 false positive execution claims in mixed configuration and 11 false positive claims in unified configuration, corresponding to hallucination rates of 12.6% and 4.7%. These results quantify the validator’s role in maintaining execution reliability and preventing the propagation of erroneous claims through the validation pipeline.

The validator’s hint generation mechanism operates through dynamic oracle construction, providing specific failure context when validation fails. Analysis of retry sequences shows that 78% of initially failed tasks achieve successful validation after incorporating validator feedback, demonstrating the iterative refinement capability of the multi-agent system.

Overall Validation Results.

Among the 82 speculative vulnerability findings from aggregation, our analysis identifies 56 true positives, 7 false positives, 17 out-of-scope cases, and 2 unable-to-install instances. Mixed configuration achieves validation of 46 out of 75 actionable cases (61.3%), while unified configuration reaches 51 out of 75 cases (68.0%), representing a +6.7 percentage point improvement. For end-to-end effectiveness based on the 60 labeled vulnerabilities in Ghera, those marked with asterisks (*) in Table LABEL:tab:poc_summary in the Appendix correspond to ground truth labels. Mixed configuration validates 31 labeled vulnerabilities (51.7%), while unified configuration achieves 34 (56.7%). This +5.0 percentage point improvement demonstrates the impact of consistent model capability across validation agents, confirming the importance of model selection in autonomous security analysis.

5.6 Cost and Efficiency Analysis

To evaluate the practical deployment feasibility of A2, we analyze computational costs and efficiency across different system configurations and operational phases.

Token Consumption Analysis.

Token consumption exhibits distinct scaling patterns across operational phases. During individual vulnerability detection (Figure 10a), models consume 1,322-1,552 prompt tokens and 778-3,523 completion tokens per APK. The aggregation phase (Figure 10b) requires 19,074-24,477 prompt tokens and 986-36,624 completion tokens, representing a 15-20x increase due to processing outputs from seven detection tools. In the multi-agent validation pipeline, the Executor component dominates consumption with median prompt tokens of 222,733 (mixed configuration, Figure 10c) and 350,667 (unified configuration, Figure 10d), while Planner and Validator components require substantially fewer resources. This distribution indicates that 70-80% of computational costs concentrate in the exploitation generation phase, suggesting optimization opportunities.

Economic Cost Assessment.

Based on model pricing in our LLM selection criteria 5.2, we estimate per-vulnerability validation costs across configurations. Detection-only costs range from $0.003-0.029 per APK (o3),$0.0004-0.001 per APK (gpt-oss-120b), to $0.002-0.014 per APK (Gemini variants). Aggregation increases costs to$0.04-0.33 per APK for gpt-oss-120b, $0.06-0.66 per APK for gemini-2.5-flash,$0.26-0.61 per APK for gemini-2.5-pro, and $0.84-3.35 per APK for o3. Full validation pipeline costs vary significantly: mixed configuration averages$0.59-4.23 per vulnerability (median $1.77), while unified gemini-2.5-pro configuration ranges$4.81-26.85 per vulnerability (median $8.94). Despite higher per-token costs, unified configuration achieves 82% reduction in Executor token usage, though this translates to only marginal cost savings due to gemini-2.5-pro’s higher pricing structure.

Efficiency Analysis.

We analyze A2’s time overhead. During Agentic Vulnerability Discovery, efficiency differences primarily stem from varying model request times since this phase involves code-level analysis without dynamic APK interaction. We focus on Agentic Vulnerability Validation time overhead. Figure 11 presents execution time distributions across both configurations. Mixed configuration achieves mean execution time of 11.14 minutes (median 8.29 minutes, $\sigma$ = 11.70), while unified configuration demonstrates 9.62 minutes mean (median 9.24 minutes, $\sigma$ = 6.44). Both configurations complete 75% of vulnerabilities within 11 minutes, indicating comparable baseline throughput. However, temporal outliers extend to 63 minutes (mixed) and 41 minutes (unified), representing $<$5% of validation cases requiring extended analysis. The unified configuration exhibits 45% reduced variance and 35% lower maximum execution time due to reduced function call requirements, enabling validation completion with fewer execution steps.

6 Real-World Vulnerability Detection

To evaluate A2’s effectiveness on real-world production applications, we conduct experiments on a real-world dataset comprising 169 APKs with an average of 115,696 lines of code per application. The experimental configuration employs (i) two gemini-2.5-pro models as detectors with gemini-2.5-flash as aggregator, and (ii) gemini-2.5-flash as planner, executor, and validator for the validation pipeline. After excluding 9 APKs that cannot be installed on the Android emulator, we analyze the remaining 160 APKs.

Detection Results.

Table 4 presents the detection results. We apply A2 to scan the 160 APKs. During the Agentic Vulnerability Discovery phase, A2 reports 136 speculative vulnerabilities. In the Agentic Vulnerability Validation phase, A2 successfully reports 60 validated vulnerability findings and filters out 29 false positive findings. Through manual review, we further identify that only 3 out of the 60 validated vulnerability findings are false positives, resulting in a false positive rate of 5.0% (3/60).

Vulnerability Classification.

Table 5 analyzes the distribution of true positive vulnerability types across discovery and validation phases. Agentic Vulnerability Validation demonstrates high effectiveness for Data Exposure vulnerabilities, validating 77.3% (17/22) of speculative findings. Code Injection vulnerabilities also show strong validation rates at 80.0% (12/15). In contrast, Network Communication vulnerabilities exhibit low validation rates at 4.3% (1/23). Similar to the Ghera dataset findings, network-related vulnerabilities involve man-in-the-middle attacks, certificate manipulation, and network traffic monitoring that require infrastructure beyond the Android emulator capabilities that A2 currently lacks.

Responsible Disclosure.

Following security research practices, we responsibly disclose identified vulnerabilities through bug bounty programs, official guidelines, or direct contact with developers. Confirmed cases are documented and reported via proper channels, with sufficient time for remediation before public release. The findings span cryptographic, input validation, and access control flaws, demonstrating A2’s broad detection capabilities in real-world Android apps. Full details will be shared after remediation.

7 Discussion

Do We Still Need SAST Tools?

Despite LLM-based analyzers achieving superior vulnerability coverage compared to individual SAST tools, traditional static analysis retains critical value within A2’s architecture. (i) Exploit Path Guidance: Static tools excel at providing code-line granular vulnerability localization and data flow analysis that enables precise source-to-sink path identification. This information, when aggregated into standardized reports, assists the PoC Planner in synthesizing targeted attack vectors through precise entry point identification. (ii) Hallucination Filtering: Call graph analysis mitigates LLM false positives when context windows approach model capacity limits, preventing phantom function invocation claims that lead to validation overhead. Consider the following anonymized code fragment from our real-world evaluation dataset (Figure 12):

Our LLM analyzer identified this as a Path Traversal vulnerability due to unsanitized string parameters directly constructing file paths, potentially leading to deleting critical databases or shared preferences files. However, call graph analysis revealed these methods lack both internal invocation and external exposure, rendering the speculative vulnerability finding a false positive despite structural vulnerability presence. While our validation framework correctly classified this as FP, integrating call graph analysis tools like AndroidGuard [26] during discovery phases could eliminate such spurious vulnerability findings, reducing validation costs.

Long-Context Function-Calling Degradation.

The Planner-Executor-Validator architecture is designed to decompose validation tasks into step-by-step operations. However, as execution sequences grow longer, we observe a degradation in function-calling accuracy, particularly when calls deviate from expected outcomes. In practice, gemini-2.5-flash and gemini-2.5-pro exhibit long-context function-calling degradation [36], where the models repeatedly invoke the same function with identical arguments, relying on local context rather than global optimization. This behavior becomes especially pronounced after about 14 execution steps in our real-world evaluation, resulting in substantial cost overhead. To mitigate this issue, future work will explore cycle-detection mechanisms that halt Task Executors once consecutive identical function calls (same function, same arguments) exceed a threshold $N$, thereby triggering summarization or alternative strategies to maintain execution efficiency.

8 Limitations

Key caveats are noted to clarify the scope of our findings:

Vulnerability Discovery Scope.

As shown in Table 6 in the Appendix, the aggregation approach achieves 78.3% recall (47/60 vulnerabilities), leaving 13 vulnerabilities undetected across all evaluation tools. These limitations fall into three primary categories: (i) Android manifest configuration vulnerabilities requiring runtime environment understanding, such as TaskAffinity-based attacks that exploit inter-application task management beyond static code analysis; (ii) runtime context dependencies exemplified by permission bypass vulnerabilities like CheckPermission-PrivilegeEscalation, which exploit timing-sensitive behavior of Binder methods when called from different thread contexts; and (iii) protocol-level implementation subtleties, including InsecureSSLSocket-MITM vulnerabilities that depend on recognizing the distinction between certificate authority validation and hostname verification in SSLCertificateSocketFactory implementations.

Vulnerability Validation Scope.

As noted in Section 5.5, 17 of 82 speculative vulnerability findings fall outside A2’s current validation capabilities. These involve cases like network traffic monitoring or man-in-the-middle attacks that require infrastructure beyond the Android emulator, such as packet interception or SSL manipulation. Since the framework relies on observable app state changes, it is less effective for vulnerabilities that manifest only through external network behavior or specialized attack setups.

LLM Reasoning Reliability.

Despite implementing multi-agent validation mechanisms, A2 exhibits residual hallucination rates ranging from 4.7% to 12.6%, as evidenced in Table LABEL:tab:poc_summary in the Appendix. The Task Validator successfully filters erroneous claims through dynamically generated oracles, yet cannot completely eliminate false positives, particularly for semantic vulnerabilities where oracle generation proves challenging. For instance, unified configuration misclassifies OpenSocket-InformationLeak-2 as a true positive, demonstrating the difficulty in programmatically determining sensitive information exposure without human judgment.

Context Window Constraints.

As noted in Section 5.1, model context limitations necessitate filtering APKs to those under 5MB during vulnerability detection phases. This constraint potentially excludes larger, more complex applications that may harbor vulnerabilities, thereby limiting the approach’s applicability to enterprise-scale mobile applications. When application complexity exceeds context window capacity, reduced analysis coverage may lead to decreased detection effectiveness and increased false negative rates.

9 Related Works

Security Analysis of Android Applications.

Android security tools employ diverse approaches to vulnerability detection. MobSF [5] combines static and dynamic analysis in an all-in-one framework, while APKHunt [6] achieves the broadest coverage (67% of OWASP MASVS categories) through exhaustive pattern matching on decompiled sources [14]. Trueseeing [35] operates directly on bytecode to resist obfuscation. However, a comprehensive study by Zhu et al. [14] revealed fundamental limitations across 11 Android SAST tools: they miss numerous vulnerability classes and generate excessive false positives. Beyond detection, Chen et al. [37] demonstrated automated attack generation through GUI-squatting that creates phishing apps. Despite these advances, current tools cannot automatically validate or exploit their findings, requiring extensive manual verification that creates a critical gap between detection and validation.

LLMs in Android Testing.

Large language models have introduced semantic reasoning to Android testing. AutoDroid [22] pioneered LLM-driven UI automation, achieving 71.3% task completion without manual scripting. LLMDroid [38] strategically invokes LLMs when traditional crawlers plateau, improving code coverage by 26%. VisionDroid [29] leverages multimodal models to detect non-crash bugs that evade traditional tools. Supporting frameworks like Guardian [39] optimize LLM computational overhead through specialized runtime systems. While these approaches significantly expand testing capabilities through intelligent navigation and input generation, they primarily target functional correctness rather than security vulnerabilities, leaving LLM-based vulnerability discovery and exploit generation largely unexplored in the Android domain.

Automated Exploit Generation.

Recent research demonstrates LLMs’ potential for autonomous exploitation across security domains [40, 41, 42]. In mobile security, AdbGPT [43] reproduces crashes from natural-language bug reports with 81.3% success. Fang et al. showed GPT-4 agents can autonomously hack websites through blind SQL injection [44], with multi-agent teams achieving 4.3× higher success rates [45]. For smart contracts, Gervais and Zhou’s A1 system [31] equips LLMs with blockchain-specific tools to generate and validate exploits end-to-end, achieving 63% success rate on vulnerable contracts with multi-million-dollar attack payloads. These studies demonstrate that domain-specific tool integration enables exploit generation, yet this capability remains unrealized for Android applications—our work bridges this gap by combining Android vulnerability detection with automated validation through concrete PoC generation.

10 Conclusion

We presented A2, a system that combines agentic discovery with validation to mirror how human experts analyze Android applications. On the Ghera benchmark, A2 achieves higher coverage than existing tools, reduces noise, and provides concrete proof-of-concept exploits. In real-world evaluation, A2 uncovered 104 previously unknown zero-day vulnerabilities, over half of which were self-validated with working exploits. By turning speculative findings into validated vulnerabilities, A2 takes a step toward practical, automated security analysis for the Android ecosystem.

11 Ethical Considerations

This research is conducted with careful attention to ethical responsibilities, which are detailed below.

Vulnerability Assessment Environment

All vulnerability assessments are conducted in controlled, isolated environments that do not involve real user data or privacy. The Ghera benchmark dataset consists of synthetically vulnerable APKs designed specifically for security research, eliminating concerns about real-world user privacy or data exposure. For production APK analysis, applications are obtained from AndroZoo, a research-oriented dataset that aggregates publicly available APKs without including user data. All testing is performed on local Android emulators in sandboxed environments, ensuring no interaction with live production systems or user devices.

Responsible Vulnerability Disclosure

Following established security research practices, all identified vulnerabilities undergo responsible disclosure procedures. Upon confirmation of vulnerabilities in production applications, affected developers and security teams are immediately contacted through appropriate channels, including official bug bounty programs, GitHub repositories, and direct developer communication. Each disclosure includes comprehensive documentation of the vulnerability details, exploitation mechanics, and recommended remediation strategies. Adequate time is provided for vulnerability remediation before any public disclosure, adhering to industry-standard timelines. No specific vulnerability details are disclosed in this paper until remediation is complete.

Dataset and Privacy Considerations

The research datasets utilized pose minimal privacy risks. The Ghera benchmark contains no real user data, consisting entirely of synthetic vulnerable applications created for research purposes. The AndroZoo dataset comprises publicly distributed applications without user-generated content or personal data. All analysis is conducted on application binaries and source code, not on user data or behavioral information. The research methodology specifically avoids any collection, processing, or analysis of personal user information.

Artifact Availability and Potential for Misuse

Any vulnerability discovery tool carries a general risk of misuse, since the same techniques that help defenders identify weaknesses could be repurposed by attackers. A2 is no exception: its ability to generate working exploits could, in principle, be abused against production applications. At the same time, this risk is not unique to our work—numerous open-source and commercial tools already provide comparable capabilities.

The intent of A2 is defensive: to help researchers and developers identify and fix vulnerabilities before they are exploited in the wild. Its novelty lies in combining discovery with automated validation, enabling more reliable security assessments. To further reduce misuse risk, artifact access is managed under a controlled distribution strategy that requires institutional affiliation and a declared research purpose, striking a balance between open science and responsible disclosure.

Broader Impact Assessment

The research contributes positively to Android application security by providing enhanced vulnerability detection capabilities that can strengthen the overall security posture of the mobile application ecosystem. By enabling more effective identification and validation of security vulnerabilities, A2 supports proactive security measures that benefit end users through improved application security. The responsible disclosure practices employed ensure that identified vulnerabilities are remediated rather than exploited maliciously.

12 Open Science

The Ghera benchmark dataset containing 60 vulnerable Android APKs is also publicly available through its official repository [32]. To mitigate potential misuse (detailed in the Ethical Considerations section), we will implement a controlled access policy. Researchers can request access by contacting us through their institutional email, stating their name, affiliation, and intended use. We will then vet the provided information and grant or deny access to the source code. Our goal is to grant access to academic and industry researchers for purposes of building upon our research or advancing the field of Android security, while ensuring A2 is not exploited for malicious purposes.

The production APK dataset comprising 169 applications from AndroZoo [11] is accessible through the AndroZoo research platform with appropriate academic credentials. For security considerations, specific application identifiers and vulnerable application details from the production dataset are not disclosed to prevent potential misuse by malicious actors, unless researchers contact us and provide appropriate academic credentials.