Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents

TL;DR

This survey reviews the technological advancements, industry applications, and evaluation methods of LLM-driven industry agents, highlighting their evolution from simple process systems to complex autonomous and social systems.

Contribution

It provides a comprehensive framework and analysis of the current state, challenges, and future directions of industry agents based on large language models.

Findings

Key technological pillars: Memory, Planning, Tool Use.

Applications span digital engineering, scientific discovery, and more.

Evaluation challenges include authenticity, safety, and industry-specific benchmarks.

Abstract

With the rise of large language models (LLMs), LLM agents capable of autonomous reasoning, planning, and executing complex tasks have become a frontier in artificial intelligence. However, how to translate the research on general agents into productivity that drives industry transformations remains a significant challenge. To address this, this paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on LLMs. Using an industry agent capability maturity framework, it outlines the evolution of agents in industry applications, from "process execution systems" to "adaptive social systems." First, we examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use. We discuss how these technologies evolve from supporting simple tasks in their early forms to enabling complex autonomous systems and collective intelligence in more advanced forms. Then, we provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation. Additionally, this paper reviews the evaluation benchmarks and methods for both fundamental and specialized capabilities, identifying the challenges existing evaluation systems face regarding authenticity, safety, and industry specificity. Finally, we focus on the practical challenges faced by industry agents, exploring their capability boundaries, developmental potential, and governance issues in various scenarios, while providing insights into future directions. By combining technological evolution with industry practices, this review aims to clarify the current state and offer a clear roadmap and theoretical foundation for understanding and building the next generation of industry agents.