Agentic Insight Generation in VSM Simulations

TL;DR

This paper introduces a two-step agentic architecture leveraging large language models to improve insight generation from complex value stream map simulations, emphasizing structured reasoning and domain knowledge.

Contribution

It presents a novel decoupled architecture that separates orchestration from data analysis, enabling better subtle difference detection in simulation data.

Findings

Top-tier models achieved up to 86% accuracy.

The framework demonstrated high robustness across evaluations.

The approach effectively integrates domain knowledge with LLMs.

Abstract

Extracting actionable insights from complex value stream map simulations can be challenging, time-consuming, and error-prone. Recent advances in large language models offer new avenues to support users with this task. While existing approaches excel at processing raw data to gain information, they are structurally unfit to pick up on subtle situational differences needed to distinguish similar data sources in this domain. To address this issue, we propose a decoupled, two-step agentic architecture. By separating orchestration from data analysis, the system leverages progressive data discovery infused with domain expert knowledge. This architecture allows the orchestration to intelligently select data sources and perform multi-hop reasoning across data structures while maintaining a slim internal context. Results from multiple state-of-the-art large language models demonstrate the framework's viability: with top-tier models achieving accuracies of up to 86% and demonstrating high robustness across evaluation runs.