Bi-Fact: A Bidirectional Factorization-based Evaluation of Intent Extraction from UI Trajectories

TL;DR

Bi-Fact is a new evaluation method that decomposes UI intents into atomic facts and uses bidirectional comparisons to improve accuracy and correlation with human judgments.

Contribution

It introduces a novel bidirectional factorization approach for intent evaluation from UI trajectories, enhancing precision and recall measurement.

Findings

Bi-Fact shows superior correlation with human judgments.

It provides a more robust evaluation framework for UI intent understanding.

Outperforms existing metrics in experiments.

Abstract

Evaluating intent extraction from GUIs demands accurate, fine-grained metrics. This paper introduces Bi-Fact, a novel method that decomposes intents into atomic facts and performs bidirectional comparisons to assess precision and recall. Experiments demonstrate Bi-Fact's superior correlation with human judgments compared to existing metrics, establishing a more robust evaluation framework for UI-driven intent understanding.