Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment

TL;DR

This paper introduces BBCritic, a contrastive learning-based GUI critic that captures hierarchical affordance structures, surpassing binary classifiers in fine-grained ranking and zero-shot transferability.

Contribution

It proposes a novel contrastive learning paradigm for GUI critique, addressing binary model limitations and introducing a hierarchical benchmark for evaluation.

Findings

BBCritic outperforms binary models with 7B parameters.

Demonstrates strong zero-shot transferability.

Introduces BBBench, a hierarchical GUI critic benchmark.

Abstract

Test-Time Scaling (TTS), which samples multiple candidate actions and ranks them via a Critic Model, has emerged as a promising paradigm for generalist GUI agents. Its efficacy thus hinges on the critic's fine-grained ranking ability. However, existing GUI critic models uniformly adopt binary classification. Our motivational analysis of these models exposes a severe entanglement: scores for valid actions and plausible-but-invalid distractors become indistinguishable. We attribute this failure to two structural defects: Affordance Collapse--the hierarchical affordance space is compressed into 0/1 labels; and Noise Sensitivity--binary objectives overfit to noisy decision boundaries. To resolve this, we introduce BBCritic (Beyond-Binary Critic), a paradigm shift grounded in the Functional Equivalence Hypothesis. Through two-stage contrastive learning, BBCritic aligns instructions and actions in a shared Affordance Space, recovering the hierarchical structure that binary supervision flattens. We also present BBBench (Beyond-Binary Bench), the first GUI critic benchmark that pairs a dense action space with a hierarchical four-level taxonomy, enabling fine-grained ranking evaluation. Experimental results show that BBCritic-3B, trained without any extra annotation, outperforms 7B-parameter SOTA binary models. It demonstrates strong zero-shot transferability across platforms and tasks, supporting our methodological view: GUI critique is fundamentally a metric-learning problem, not a classification one.