GEM: Gaussian Embedding Modeling for Out-of-Distribution Detection in GUI Agents

TL;DR

GEM is a Gaussian mixture model-based method that improves out-of-distribution detection in GUI agents by analyzing input embedding distances, leading to higher accuracy and better task success rates across diverse environments.

Contribution

This work introduces GEM, a novel Gaussian mixture model approach that effectively detects OOD inputs in GUI agents by leveraging clustering patterns in embedding space.

Findings

Achieves 23.70% accuracy improvement over baselines

Increases step-wise success rate by 9.40% with cloud assistance

Demonstrates strong generalization across nine backbone models

Abstract

Graphical user interface (GUI) agents have recently emerged as an intriguing paradigm for human-computer interaction, capable of automatically executing user instructions to operate intelligent terminal devices. However, when encountering out-of-distribution (OOD) instructions that violate environmental constraints or exceed the current capabilities of agents, GUI agents may suffer task breakdowns or even pose security threats. Therefore, effective OOD detection for GUI agents is essential. Traditional OOD detection methods perform suboptimally in this domain due to the complex embedding space and evolving GUI environments. In this work, we observe that the in-distribution input semantic space of GUI agents exhibits a clustering pattern with respect to the distance from the centroid. Based on the finding, we propose GEM, a novel method based on fitting a Gaussian mixture model over input embedding distances extracted from the GUI agent that reflect its capability boundary. Evaluated on eight datasets spanning smartphones, computers, and web browsers, our method achieves an average accuracy improvement of 23.70\% over the best-performing baseline while only increasing training time by 4.9\% and testing time by 6.5\%. We also experimentally demonstrate that GEM can improve the step-wise success rate by 9.40\% by requesting assistance from the cloud model when encountering OOD samples. Analysis verifies the generalization ability of our method through experiments on nine different backbones. The codes are available at https://github.com/Wuzheng02/GEM-OODforGUIagents.