Reliable agent engineering should integrate machine-compatible organizational principles

TL;DR

This paper explores how principles from organizational science can inform the design and management of reliable AI agents built on large language models, emphasizing balancing capabilities, resource constraints, and governance mechanisms.

Contribution

It introduces organizational principles tailored for AI agent engineering, bridging social organization theories with AI system reliability and effectiveness.

Findings

Organizational principles can enhance AI agent reliability.

Balancing agency and capabilities improves system robustness.

Resource management strategies are crucial for scalable AI agents.

Abstract

As AI agents built on large language models (LLMs) become increasingly embedded in society, issues of coordination, control, delegation, and accountability are entangled with concerns over their reliability. To design and implement LLM agents around reliable operations, we should consider the task complexity in the application settings and reduce their limitations while striving to minimize agent failures and optimize resource efficiency. High-functioning human organizations have faced similar balancing issues, which led to evidence-based theories that seek to understand their functioning strategies. We examine the parallels between LLM agents and the compatible frameworks in organization science, focusing on what the design, scaling, and management of organizations can inform agentic systems towards improving reliability. We offer three preliminary accounts of organizational principles for AI agent engineering to attain reliability and effectiveness, through balancing agency and capabilities in agent design, resource constraints and performance benefits in agent scaling, and internal and external mechanisms in agent management. Our work extends the growing exchanges between the operational and governance principles of AI systems and social systems to facilitate system integration.