No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows

TL;DR

MOSAIC is a training-free multi-agent framework that generates scientific workflows without I/O test cases, using knowledge distillation and structured problem decomposition to improve accuracy and consistency.

Contribution

It introduces MOSAIC, a novel approach for scientific code generation that does not require I/O supervision and mitigates hallucinations with a Consolidated Context Window.

Findings

MOSAIC outperforms existing methods on the SciCode benchmark.

It improves accuracy, executability, and numerical precision.

Uses lightweight models with a knowledge distillation framework.

Abstract

Existing multi-agent Large Language Model (LLM) frameworks for code generation typically use execution feedback and improve iteratively using Input/Output (I/O) test cases. However, this does not work for scientific workflows, where I/O test cases do not exist, and generating them requires solving the very problem at hand. To address this, we introduce MOSAIC, a training-free multi-agent framework for scientific code generation without I/O supervision. Instead of execution feedback, MOSAIC employs a student-teacher knowledge distillation framework that grounds generation through domain-specific examples and structured problem decomposition. To further mitigate hallucinations across chained subproblems, we introduce a Consolidated Context Window (CCW) for maintaining consistent reasoning across agents. Experiments on the SciCode benchmark show that MOSAIC improves accuracy, executability, and numerical precision over existing approaches while relying on lightweight models.