From Language Models to Practical Self-Improving Computer Agents

TL;DR

This paper presents a methodology for creating self-improving AI agents that can generate and utilize software augmentations to perform complex real-world tasks, reducing manual engineering effort.

Contribution

It introduces a systematic approach for LLM agents to autonomously generate and employ software tools for self-augmentation, enabling scalable task solving.

Findings

LLM agents can autonomously develop tools like retrieval and web navigation.

Self-augmentation improves the agent's ability to perform complex tasks.

Minimal prompting enables effective self-improvement in real-world scenarios.

Abstract

We develop a simple and straightforward methodology to create AI computer agents that can carry out diverse computer tasks and self-improve by developing tools and augmentations to enable themselves to solve increasingly complex tasks. As large language models (LLMs) have been shown to benefit from non-parametric augmentations, a significant body of recent work has focused on developing software that augments LLMs with various capabilities. Rather than manually developing static software to augment LLMs through human engineering effort, we propose that an LLM agent can systematically generate software to augment itself. We show, through a few case studies, that a minimal querying loop with appropriate prompt engineering allows an LLM to generate and use various augmentations, freely extending its own capabilities to carry out real-world computer tasks. Starting with only terminal access, we prompt an LLM agent to augment itself with retrieval, internet search, web navigation, and text editor capabilities. The agent effectively uses these various tools to solve problems including automated software development and web-based tasks.