SecAgent: Efficient Mobile GUI Agent with Semantic Context
Yiping Xie, Song Chen, Jingxuan Xing, Wei Jiang, Zekun Zhu, Yingyao Wang, Pi Bu, Jun Song, Yuning Jiang, Bo Zheng

TL;DR
SecAgent is an efficient 3B-scale mobile GUI agent that leverages a new Chinese dataset and semantic context summarization to improve task automation while reducing computational costs.
Contribution
The paper introduces a Chinese mobile GUI dataset, a semantic context mechanism, and demonstrates that SecAgent outperforms similar-scale models on navigation benchmarks.
Findings
SecAgent achieves performance comparable to larger models (7B-8B) on navigation tasks.
The semantic context mechanism reduces computational costs significantly.
The dataset includes 18k grounding samples and 121k navigation steps across 44 apps.
Abstract
Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the scarcity of high-quality multilingual datasets, particularly for non-English ecosystems, and inefficient history representation methods. To address these challenges, we present SecAgent, an efficient mobile GUI agent at 3B scale. We first construct a human-verified Chinese mobile GUI dataset with 18k grounding samples and 121k navigation steps across 44 applications, along with a Chinese navigation benchmark featuring multi-choice action annotations. Building upon this dataset, we propose a semantic context mechanism that distills history screenshots and actions into concise, natural language summaries, significantly reducing computational costs while…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
