Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization

TL;DR

This paper demonstrates that simple AI agents can effectively automate the adaptation of computer vision tools for biomedical imaging, outperforming human experts and providing a practical framework for real-world deployment.

Contribution

Introduces a systematic evaluation framework for agentic code optimization and shows that simple agents outperform experts in biomedical imaging pipeline adaptation.

Findings

Simple agents outperform human experts in pipeline adaptation

Complex agent architectures are not always more effective

Open source framework facilitates real-world deployment

Abstract

Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck. Current solutions are impractical: fine-tuning requires large annotated datasets scientists often lack, while manual code adaptation costs scientists weeks to months of effort. We consider using AI agents to automate this manual coding, and focus on the open question of optimal agent design for this targeted task. We introduce a systematic evaluation framework for agentic code optimization and use it to study three production-level biomedical imaging pipelines. We demonstrate that a simple agent framework consistently generates adaptation code that outperforms human-expert solutions. Our analysis reveals that common, complex agent architectures are not universally beneficial, leading to a practical roadmap for agent design. We open source our framework and validate our approach by deploying agent-generated functions into a production pipeline, demonstrating a clear pathway for real-world impact.