An Open Source Computer Vision and Machine Learning Framework for Affordable Life Science Robotic Automation

TL;DR

This paper introduces an open-source, low-cost robotic framework integrating computer vision and machine learning for automation in life sciences, enabling tasks like colony picking and liquid handling with high precision.

Contribution

The framework combines a custom-trained U-net for microbial segmentation and a Mixture Density Network for inverse kinematics, providing an affordable automation solution.

Findings

Mean positional error below 1 mm

Joint angle prediction errors below 4 degrees

Colony detection IoU score of 0.537

Abstract

We present an open-source robotic framework that integrates computer vision and machine learning based inverse kinematics to enable low-cost laboratory automation tasks such as colony picking and liquid handling. The system uses a custom trained U-net model for semantic segmentation of microbial cultures, combined with Mixture Density Network for predicating joint angles of a simple 5-DOF robot arm. We evaluated the framework using a modified robot arm, upgraded with a custom liquid handling end-effector. Experimental results demonstrate the framework's feasibility for precise, repeatable operations, with mean positional error below 1 mm and joint angle prediction errors below 4 degrees and colony detection capabilities with IoU score of 0.537 and Dice coefficient of 0.596.