REX: Rapid Exploration and eXploitation for AI Agents

TL;DR

REX introduces a reward-based framework for AI agents that improves exploration and exploitation efficiency, surpassing some existing methods in performance and significantly reducing execution time without requiring model fine-tuning.

Contribution

The paper presents REX, a novel approach that integrates reward mechanisms and UCB-like scores into AI agents, enabling more robust, efficient, and fine-tuning-free exploration and exploitation.

Findings

REX achieves comparable or better performance than Chain-of-Thoughts and RAP.

REX significantly reduces execution time of AI agents.

REX effectively leverages offline logs and integrates with foundation models.

Abstract

In this paper, we propose an enhanced approach for Rapid Exploration and eXploitation for AI Agents called REX. Existing AutoGPT-style techniques have inherent limitations, such as a heavy reliance on precise descriptions for decision-making, and the lack of a systematic approach to leverage try-and-fail procedures akin to traditional Reinforcement Learning (RL). REX introduces an additional layer of rewards and integrates concepts similar to Upper Confidence Bound (UCB) scores, leading to more robust and efficient AI agent performance. This approach has the advantage of enabling the utilization of offline behaviors from logs and allowing seamless integration with existing foundation models while it does not require any model fine-tuning. Through comparative analysis with existing methods such as Chain-of-Thoughts(CoT) and Reasoning viA Planning(RAP), REX-based methods demonstrate comparable performance and, in certain cases, even surpass the results achieved by these existing techniques. Notably, REX-based methods exhibit remarkable reductions in execution time, enhancing their practical applicability across a diverse set of scenarios.