CRAFT-GUI: Curriculum-Reinforced Agent For GUI Tasks

TL;DR

CRAFT-GUI introduces a curriculum reinforcement learning framework that improves GUI task performance by accounting for task difficulty variation and providing nuanced reward signals, leading to significant benchmarks improvements.

Contribution

It presents a novel curriculum learning approach with a new reward function for reinforcement learning in GUI tasks, addressing prior limitations of uniform training data and coarse rewards.

Findings

Achieves 5.6% improvement on Android Control benchmark.

Achieves 10.3% improvement on internal benchmarks.

Validates effectiveness of curriculum RL in GUI tasks.

Abstract

As autonomous agents become adept at understanding and interacting with graphical user interface (GUI) environments, a new era of automated task execution is emerging. Recent studies have demonstrated that Reinforcement Learning (RL) can effectively enhance agents' performance in dynamic interactive GUI environments. However, these methods face two key limitations: (1) they overlook the significant variation in difficulty across different GUI tasks by treating the entire training data as a uniform set, which hampers the agent's ability to adapt its learning process; and (2) most approaches collapse task-specific nuances into a single, coarse reward, leaving the agent with a uniform signal that yields inefficient policy updates. To address these limitations, we propose CRAFT-GUI, a curriculum learning framework based on Group Relative Policy Optimization (GRPO) that explicitly accounts for the varying difficulty across trajectories. To enable more fine-grained policy optimization, we design a reward function that combines simple rule-based signals with model-judged evaluation, providing richer and more nuanced feedback during training. Experimental results demonstrate that our method achieves significant improvements over previous state-of-the-art approaches, outperforming them by 5.6% on public benchmarks Android Control and 10.3% on our internal online benchmarks, respectively. These findings empirically validate the effectiveness of integrating reinforcement learning with curriculum learning in GUI interaction tasks.