Abstract:Language models can produce convincing scientific analyses, but repeated generations on the same data do not guarantee the same result. A researcher may regenerate an identical query and receive a different fit, a different peak position or a different analysis procedure, without an obvious way to decide which output to trust. We propose typed mediation, a pattern in which the model orchestrates deterministic tools rather than generating analytical code. Each tool encodes one researcher's exact procedure for one instrument, ported through structured interviews. The model selects which tool to call and with what parameters. The tool produces the result. Regeneration does not change it. We evaluate this claim by running the same photoluminescence analysis on four platforms, including three commercial foundation models, four times each with the same prompt. The typed tool produces identical results across all runs. The commercial platforms either vary in numerical output and analytical methodology across runs, or fail to produce valid results on the task. We deploy this pattern on two instruments serving users over approximately six months, with very positive user feedback. Both cases are very challenging: they involve proprietary binary formats and per-seat licensed software, which force the tool to remain on local infrastructure alongside the data and the instrument it operates. We argue that deployment topology is not just a preference, but a structural requirement of scientific tool mediation. The result is a practical pattern for deploying language models in scientific workflows where reproducibility is mandatory, reducing analysis time from weeks to minutes while guaranteeing identical outputs across runs.
| Comments: | 18 pages, 4 figures, 2 appendices. Submitted to SETN 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.1; J.2 |
| Cite as: | arXiv:2605.13245 [cs.AI] |
| (or arXiv:2605.13245v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13245 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Marios Adamidis [view email]
[v1]
Wed, 13 May 2026 09:30:56 UTC (5,774 KB)