A Survey on Complex Tasks for Goal-Directed Interactive Agents

TL;DR

This survey reviews complex tasks and environments used to evaluate goal-directed interactive agents, highlighting challenges and resources to advance agent development and understanding.

Contribution

It provides a comprehensive compilation and structured analysis of tasks and environments for evaluating goal-directed interactive agents.

Findings

Identifies key challenges in current evaluation tasks

Organizes tasks along relevant dimensions

Provides resources for future research

Abstract

Goal-directed interactive agents, which autonomously complete tasks through interactions with their environment, can assist humans in various domains of their daily lives. Recent advances in large language models (LLMs) led to a surge of new, more and more challenging tasks to evaluate such agents. To properly contextualize performance across these tasks, it is imperative to understand the different challenges they pose to agents. To this end, this survey compiles relevant tasks and environments for evaluating goal-directed interactive agents, structuring them along dimensions relevant for understanding current obstacles. An up-to-date compilation of relevant resources can be found on our project website: https://coli-saar.github.io/interactive-agents.