EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning

TL;DR

This paper introduces EE-MCP, a self-evolving framework for MCP-GUI agents that automatically generates environments, learns from experience, and improves performance across diverse applications without manual intervention.

Contribution

It presents a novel unified hybrid policy learning approach and an experience bank mechanism for self-improvement in MCP-GUI agents, enabling application-aware modality balancing.

Findings

Distillation achieves 77.8% pass rate on MCP-dominant tasks (+17.8pp).

Experience bank improves performance on GUI-intensive tasks (+10.0pp).

The optimal strategy varies depending on MCP-GUI composition.

Abstract

Computer-use agents that combine GUI interaction with structured API calls via the Model Context Protocol (MCP) show promise for automating software tasks. However, existing approaches lack a principled understanding of how agents should balance these two modalities and how to enable iterative self-improvement across diverse applications. We formulate MCP-GUI interplay as a unified hybrid policy learning problem where the agent learns when each modality provides complementary advantages, and show that distillation and experience augmentation target fundamentally different failure modes - requiring application-aware mechanism selection. Built on this formulation, we propose a self-evolving framework with a fully automatic pipeline that orchestrates automatic environment generation and validation, trajectory collection, gap-driven task synthesis, and quality-filtered training - all without manual intervention. A key innovation is our experience bank, which accumulates LLM-learned rules from trajectory comparison, enabling inference-time improvement without fine-tuning. Systematic \textbf{cross-application analysis} across three desktop applications reveals that the optimal strategy depends on MCP-GUI composition: distillation achieves 77.8\% pass rate on MCP-dominant tasks (+17.8pp), while the experience bank excels on GUI-intensive tasks (+10.0pp).