POINTS-GUI-G: GUI-Grounding Journey
Zhongyin Zhao, Yuan Liu, Yikun Liu, Haicheng Wang, Le Tian, Xiao Zhou, Yangxiu You, Zilin Yu, Yang Yu, Jie Zhou

TL;DR
This paper introduces POINTS-GUI-G-8B, a state-of-the-art GUI grounding model that leverages refined data engineering, improved training strategies, and reinforcement learning to achieve high accuracy in locating interface elements for automation tasks.
Contribution
It presents a comprehensive pipeline starting from minimal grounding ability to achieve top performance, integrating data unification, advanced training, and RL for GUI grounding.
Findings
Achieved state-of-the-art scores on multiple GUI datasets.
Reinforcement learning significantly improves grounding precision.
Unified diverse datasets with augmentation and filtering strategies.
Abstract
The rapid advancement of vision-language models has catalyzed the emergence of GUI agents, which hold immense potential for automating complex tasks, from online shopping to flight booking, thereby alleviating the burden of repetitive digital workflows. As a foundational capability, GUI grounding is typically established as a prerequisite for end-to-end task execution. It enables models to precisely locate interface elements, such as text and icons, to perform accurate operations like clicking and typing. Unlike prior works that fine-tune models already possessing strong spatial awareness (e.g., Qwen3-VL), we aim to master the full technical pipeline by starting from a base model with minimal grounding ability, such as POINTS-1.5. We introduce POINTS-GUI-G-8B, which achieves state-of-the-art performance with scores of 59.9 on ScreenSpot-Pro, 66.0 on OSWorld-G, 95.7 on ScreenSpot-v2, and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Advanced Neural Network Applications · Topic Modeling
