Data-Centric Visual Development for Self-Driving Labs

TL;DR

This paper presents a hybrid data generation pipeline combining real and virtual images to train robust models for bubble detection in self-driving laboratories, significantly reducing data collection effort while maintaining high accuracy.

Contribution

It introduces a novel hybrid pipeline that fuses real human-in-the-loop data with reference-guided virtual data to address data scarcity in precision-sensitive tasks.

Findings

Model trained on real data achieves 99.6% accuracy.

Mixing real and virtual data maintains 99.4% accuracy.

The approach reduces data collection and review effort.

Abstract

Self-driving laboratories offer a promising path toward reducing the labor-intensive, time-consuming, and often irreproducible workflows in the biological sciences. Yet their stringent precision requirements demand highly robust models whose training relies on large amounts of annotated data. However, this kind of data is difficult to obtain in routine practice, especially negative samples. In this work, we focus on pipetting, the most critical and precision sensitive action in SDLs. To overcome the scarcity of training data, we build a hybrid pipeline that fuses real and virtual data generation. The real track adopts a human-in-the-loop scheme that couples automated acquisition with selective human verification to maximize accuracy with minimal effort. The virtual track augments the real data using reference-conditioned, prompt-guided image generation, which is further screened and validated for reliability. Together, these two tracks yield a class-balanced dataset that enables robust bubble detection training. On a held-out real test set, a model trained entirely on automatically acquired real images reaches 99.6% accuracy, and mixing real and generated data during training sustains 99.4% accuracy while reducing collection and review load. Our approach offers a scalable and cost-effective strategy for supplying visual feedback data to SDL workflows and provides a practical solution to data scarcity in rare event detection and broader vision tasks.