Can Coding Agents Be General Agents?

TL;DR

This paper investigates whether coding agents can be generalized to automate complex business processes, identifying current limitations and challenges in domain logic integration.

Contribution

The study evaluates a coding agent's performance on practical business tasks, highlighting key bottlenecks in achieving true generalization.

Findings

Agent reliably completes simple tasks

Exhibits failures on complex tasks

Bridging domain logic and code execution is a key challenge

Abstract

As coding agents have seen rapid capability and adoption gains, users are applying them to general tasks beyond software engineering. In this post, we investigate whether coding agents can successfully generalize to end-to-end business process automation. We identify gaps in current evaluations, and conduct a case study to evaluate a coding agent on practical business tasks in an open-core Enterprise Resource Planning system. We find that the agent reliably completes simple tasks but exhibits characteristic failures on complex tasks, suggesting that bridging domain logic and code execution is a key bottleneck to generalizability.