Routine: A Structural Planning Framework for LLM Agent System in Enterprise

TL;DR

Routine is a structured planning framework that significantly improves the stability and accuracy of multi-step tool-calling in enterprise LLM agent systems, facilitating practical deployment and domain adaptation.

Contribution

This paper introduces Routine, a novel multi-step agent planning framework with explicit instructions and parameter passing, enhancing stability and accuracy in enterprise LLM agent systems.

Findings

GPT-4o tool call accuracy increased from 41.1% to 96.3%.

Qwen3-14B accuracy improved from 32.6% to 83.3%.

Fine-tuning with Routine-based datasets further increased accuracy to 95.5%.

Abstract

The deployment of agent systems in an enterprise environment is often hindered by several challenges: common models lack domain-specific process knowledge, leading to disorganized plans, missing key tools, and poor execution stability. To address this, this paper introduces Routine, a multi-step agent planning framework designed with a clear structure, explicit instructions, and seamless parameter passing to guide the agent's execution module in performing multi-step tool-calling tasks with high stability. In evaluations conducted within a real-world enterprise scenario, Routine significantly increases the execution accuracy in model tool calls, increasing the performance of GPT-4o from 41.1% to 96.3%, and Qwen3-14B from 32.6% to 83.3%. We further constructed a Routine-following training dataset and fine-tuned Qwen3-14B, resulting in an accuracy increase to 88.2% on scenario-specific evaluations, indicating improved adherence to execution plans. In addition, we employed Routine-based distillation to create a scenario-specific, multi-step tool-calling dataset. Fine-tuning on this distilled dataset raised the model's accuracy to 95.5%, approaching GPT-4o's performance. These results highlight Routine's effectiveness in distilling domain-specific tool-usage patterns and enhancing model adaptability to new scenarios. Our experimental results demonstrate that Routine provides a practical and accessible approach to building stable agent workflows, accelerating the deployment and adoption of agent systems in enterprise environments, and advancing the technical vision of AI for Process.