A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learning

TL;DR

This survey reviews and unifies various frameworks for LLM-based agents focusing on tool use, planning, and feedback learning, highlighting their workflows, taxonomy, and future challenges.

Contribution

It introduces a unified taxonomy for comparing LLM-based agent frameworks across paradigms, addressing inconsistencies and identifying future research directions.

Findings

Unified taxonomy for LLM agent frameworks

Comparison of workflows and implementations across paradigms

Identification of key limitations and future research directions

Abstract

Tool use, planning, and feedback learning are currently three prominent paradigms for developing Large Language Model (LLM)-based agents across various tasks. Although numerous frameworks have been devised for each paradigm, their intricate workflows and inconsistent taxonomy create challenges in understanding and reviewing the frameworks across different paradigms. This survey introduces a unified taxonomy to systematically review and discuss these frameworks. Specifically, 1) the taxonomy defines environments/tasks, common LLM-profiled roles or LMPRs (policy models, evaluators, and dynamic models), and universally applicable workflows found in prior work, and 2) it enables a comparison of key perspectives on the implementations of LMPRs and workflow designs across different agent paradigms and frameworks. 3) Finally, we identify three limitations in existing workflow designs and systematically discuss the future work. Resources have been made publicly available at in our GitHub repository https://github.com/xinzhel/LLM-Agent-Survey.