A Dataset and Benchmarks for Deep Learning-Based Optical Microrobot Pose and Depth Perception

TL;DR

This paper introduces the OTMR dataset, a large collection of images for training and benchmarking deep learning models to accurately perceive the pose and depth of optical microrobots under microscopes, addressing key challenges in biomedicine applications.

Contribution

The paper presents the first publicly available dataset for microrobot perception, benchmarks multiple deep learning models, and demonstrates the impact of dataset size and architecture on performance.

Findings

Vision Transformer achieves highest pose classification accuracy.

Deeper architectures improve depth regression performance.

Larger training datasets significantly enhance model accuracy.

Abstract

Optical microrobots, manipulated via optical tweezers (OT), have broad applications in biomedicine. However, reliable pose and depth perception remain fundamental challenges due to the transparent or low-contrast nature of the microrobots, as well as the noisy and dynamic conditions of the microscale environments in which they operate. An open dataset is crucial for enabling reproducible research, facilitating benchmarking, and accelerating the development of perception models tailored to microscale challenges. Standardised evaluation enables consistent comparison across algorithms, ensuring objective benchmarking and facilitating reproducible research. Here, we introduce the OpTical MicroRobot dataset (OTMR), the first publicly available dataset designed to support microrobot perception under the optical microscope. OTMR contains 232,881 images spanning 18 microrobot types and 176 distinct poses. We benchmarked the performance of eight deep learning models, including architectures derived via neural architecture search (NAS), on two key tasks: pose classification and depth regression. Results indicated that Vision Transformer (ViT) achieve the highest accuracy in pose classification, while depth regression benefits from deeper architectures. Additionally, increasing the size of the training dataset leads to substantial improvements across both tasks, highlighting OTMR's potential as a foundational resource for robust and generalisable microrobot perception in complex microscale environments.