Asynchronous Tool Usage for Real-Time Agents

TL;DR

This paper introduces asynchronous AI agents that enable real-time, multitasking interactions by leveraging an event-driven architecture, improving upon traditional synchronous, turn-based systems.

Contribution

The paper presents a novel asynchronous, event-driven framework for AI agents, allowing parallel processing and real-time tool use, inspired by real-time operating systems.

Findings

Developed an event-driven finite-state machine architecture for AI agents.

Integrated automatic speech recognition and text-to-speech for real-time interaction.

Demonstrated improved multitasking capabilities in AI agents.

Abstract

While frontier large language models (LLMs) are capable tool-using agents, current AI systems still operate in a strict turn-based fashion, oblivious to passage of time. This synchronous design forces user queries and tool-use to occur sequentially, preventing the systems from multitasking and reducing interactivity. To address this limitation, we introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting, integrated with automatic speech recognition and text-to-speech. Drawing inspiration from the concepts originally developed for real-time operating systems, this work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.