A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDP

TL;DR

This paper introduces a lightweight, model-agnostic framework called DT-MDP-CE that enhances enterprise AI agents by combining digital-twin MDP modeling, contrastive inverse reinforcement learning, and RL-guided context engineering, leading to improved decision-making.

Contribution

The paper presents a novel, integrated framework for offline reinforcement learning that improves enterprise AI agents by modeling reasoning as DT-MDP and estimating reward functions from offline data.

Findings

Significant performance improvements over baseline agents.

Framework generalizes across enterprise AI tasks.

Effective in complex real-world reasoning scenarios.

Abstract

Despite rapid progress in AI agents for enterprise automation and decision-making, their real-world deployment and further performance gains remain constrained by limited data quality and quantity, complex real-world reasoning demands, difficulties with self-play, and the lack of reliable feedback signals. To address these challenges, we propose a lightweight, model-agnostic framework for improving LLM-based enterprise agents via offline reinforcement learning (RL). The proposed Context Engineering via DT-MDP (DT-MDP-CE) framework comprises three key components: (1) A Digital-Twin Markov Decision Process (DT-MDP), which abstracts the agent's reasoning behavior as a finite MDP; (2) A robust contrastive inverse RL, which, armed with the DT-MDP, to efficiently estimate a well-founded reward function and induces policies from mixed-quality offline trajectories; and (3) RL-guided context engineering, which uses the policy obtained from the integrated process of (1) and (2), to improve the agent's decision-making behavior. As a case study, we apply the framework to a representative task in the enterprise-oriented domain of IT automation. Extensive experimental results demonstrate consistent and significant improvements over baseline agents across a wide range of evaluation settings, suggesting that the framework can generalize to other agents sharing similar characteristics in enterprise environments.