Automatically Prepare Training Data for YOLO Using Robotic In-Hand Observation and Synthesis

TL;DR

This paper introduces an automated method for generating training data for object detection using robotic in-hand observation and image synthesis, reducing manual effort and cost.

Contribution

It combines robotic data collection with synthetic image generation to efficiently produce training datasets for YOLO, improving data efficiency and reducing manual labeling.

Findings

Synthetic and collected images achieve comparable detection performance.

The method reduces manual labeling effort significantly.

Optimized data configurations improve detection accuracy.

Abstract

Deep learning methods have recently exhibited impressive performance in object detection. However, such methods needed much training data to achieve high recognition accuracy, which was time-consuming and required considerable manual work like labeling images. In this paper, we automatically prepare training data using robots. Considering the low efficiency and high energy consumption in robot motion, we proposed combining robotic in-hand observation and data synthesis to enlarge the limited data set collected by the robot. We first used a robot with a depth sensor to collect images of objects held in the robot's hands and segment the object pictures. Then, we used a copy-paste method to synthesize the segmented objects with rack backgrounds. The collected and synthetic images are combined to train a deep detection neural network. We conducted experiments to compare YOLOv5x detectors trained with images collected using the proposed method and several other methods. The results showed that combined observation and synthetic images led to comparable performance to manual data preparation. They provided a good guide on optimizing data configurations and parameter settings for training detectors. The proposed method required only a single process and was a low-cost way to produce the combined data. Interested readers may find the data sets and trained models from the following GitHub repository: github.com/wrslab/tubedet