It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows

TL;DR

This paper introduces typed mediation, a pattern where language models orchestrate deterministic scientific tools to ensure reproducible results, demonstrated through photoluminescence analysis across multiple platforms.

Contribution

The paper proposes a novel pattern for scientific workflows that guarantees reproducibility by mediating deterministic tools through language models, addressing variability in analysis outputs.

Findings

Typed mediation produces identical results across multiple runs.

Commercial platforms show variability or fail to produce valid results.

Deployment on local infrastructure ensures reproducibility and user satisfaction.

Abstract

Language models can produce convincing scientific analyses, but repeated generations on the same data do not guarantee the same result. A researcher may regenerate an identical query and receive a different fit, a different peak position or a different analysis procedure, without an obvious way to decide which output to trust. We propose typed mediation, a pattern in which the model orchestrates deterministic tools rather than generating analytical code. Each tool encodes one researcher's exact procedure for one instrument, ported through structured interviews. The model selects which tool to call and with what parameters. The tool produces the result. Regeneration does not change it. We evaluate this claim by running the same photoluminescence analysis on four platforms, including three commercial foundation models, four times each with the same prompt. The typed tool produces identical results across all runs. The commercial platforms either vary in numerical output and analytical methodology across runs, or fail to produce valid results on the task. We deploy this pattern on two instruments serving users over approximately six months, with very positive user feedback. Both cases are very challenging: they involve proprietary binary formats and per-seat licensed software, which force the tool to remain on local infrastructure alongside the data and the instrument it operates. We argue that deployment topology is not just a preference, but a structural requirement of scientific tool mediation. The result is a practical pattern for deploying language models in scientific workflows where reproducibility is mandatory, reducing analysis time from weeks to minutes while guaranteeing identical outputs across runs.