Small Object Detection for Near Real-Time Egocentric Perception in a Manual Assembly Scenario

TL;DR

This paper presents a near real-time small object detection method for egocentric AR in industrial assembly, leveraging synthetic data from CAD models and a two-stage YOLOv4 approach to improve detection accuracy.

Contribution

It introduces a pipeline that uses CAD-based synthetic training data and a two-stage detection process tailored for egocentric AR scenarios.

Findings

Effective small object detection in near real-time

Utilizes synthetic data from CAD models for training

Two-stage detection improves accuracy in industrial AR

Abstract

Detecting small objects in video streams of head-worn augmented reality devices in near real-time is a huge challenge: training data is typically scarce, the input video stream can be of limited quality, and small objects are notoriously hard to detect. In industrial scenarios, however, it is often possible to leverage contextual knowledge for the detection of small objects. Furthermore, CAD data of objects are typically available and can be used to generate synthetic training data. We describe a near real-time small object detection pipeline for egocentric perception in a manual assembly scenario: We generate a training data set based on CAD data and realistic backgrounds in Unity. We then train a YOLOv4 model for a two-stage detection process: First, the context is recognized, then the small object of interest is detected. We evaluate our pipeline on the augmented reality device Microsoft Hololens 2.