MIRAGE: Online LLM Simulation for Microservice Dependency Testing

TL;DR

MIRAGE introduces an online LLM-based simulation method for microservice dependencies, achieving high fidelity in testing scenarios by dynamically generating responses based on source code and traces.

Contribution

This paper presents MIRAGE, a novel runtime approach using LLMs for dependency simulation that outperforms static artifact methods in fidelity and coverage.

Findings

MIRAGE achieves 99% fidelity in status-code and response-shape.

Static artifact methods have 0-12% fidelity in error scenarios.

Dependency source code alone suffices for high-fidelity simulation.

Abstract

Existing approaches to microservice dependency simulation--record-replay, pattern-mining, and specification-driven stubs--generate static artifacts before test execution. These artifacts can only reproduce behaviors encoded at generation time; on error-handling and code-reasoning scenarios, which are underrepresented in typical trace corpora, record-replay achieves 0% and 12% fidelity in our evaluation. We propose online LLM simulation, a runtime approach where the LLM answers each dependency request as it arrives, maintaining cross-request state throughout a test scenario. The model reads the dependency's source code, caller code, and production traces, then simulates behavior on demand--trading latency (~3 s per request) and cost ($0.16-$0.82 per dependency) for coverage on scenarios that static artifacts miss. We instantiate this approach in MIRAGE and evaluate it on 110 test scenarios across three microservice systems (Google's Online Boutique, Weaveworks' Sock Shop, and a custom system). In white-box mode, MIRAGE achieves 99% status-code and 99% response-shape fidelity, compared to 62% / 16% for record-replay. A signal ablation shows dependency source code is often sufficient (100% alone); without it, the model retains error-code accuracy (94%) but loses response-structure fidelity (75%). Results are stable across three LLM families (within 3%) and deterministic across repeated runs. Caller integration tests produce the same pass/fail outcomes with MIRAGE as with real dependencies (8/8 scenarios).