AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistant

TL;DR

AgentStore is a scalable platform that dynamically integrates heterogeneous agents, enhancing their generalization and specialization capabilities for automating complex computer tasks in real-world environments.

Contribution

It introduces a novel scalable system with a core MetaAgent and AgentToken strategy, enabling dynamic integration and management of diverse agents for improved task automation.

Findings

Significant performance improvement on OSWorld benchmark (11.21% to 23.85%)

Outperforms previous systems in generalization and specialization

Demonstrates effectiveness in complex, real-world computer tasks

Abstract

Digital agents capable of automating complex computer tasks have attracted considerable attention due to their immense potential to enhance human-computer interaction. However, existing agent methods exhibit deficiencies in their generalization and specialization capabilities, especially in handling open-ended computer tasks in real-world environments. Inspired by the rich functionality of the App store, we present AgentStore, a scalable platform designed to dynamically integrate heterogeneous agents for automating computer tasks. AgentStore empowers users to integrate third-party agents, allowing the system to continuously enrich its capabilities and adapt to rapidly evolving operating systems. Additionally, we propose a novel core \textbf{MetaAgent} with the \textbf{AgentToken} strategy to efficiently manage diverse agents and utilize their specialized and generalist abilities for both domain-specific and system-wide tasks. Extensive experiments on three challenging benchmarks demonstrate that AgentStore surpasses the limitations of previous systems with narrow capabilities, particularly achieving a significant improvement from 11.21\% to 23.85\% on the OSWorld benchmark, more than doubling the previous results. Comprehensive quantitative and qualitative results further demonstrate AgentStore's ability to enhance agent systems in both generalization and specialization, underscoring its potential for developing the specialized generalist computer assistant. All our codes will be made publicly available in https://chengyou-jia.github.io/AgentStore-Home.