Anomaly Triplet-Net: Progress Recognition Model Using Deep Metric Learning Considering Occlusion for Manual Assembly Work

TL;DR

This paper introduces Anomaly Triplet-Net, a deep metric learning model that estimates assembly progress from images despite occlusion, achieving over 82% success in factory environment experiments.

Contribution

It proposes a novel Anomaly Triplet-Net model incorporating anomaly samples for improved progress recognition under occlusion conditions.

Findings

Achieved 82.9% success rate in progress estimation.

Validated the effectiveness of detection, cropping, and estimation sequence.

Demonstrated practical applicability in factory assembly scenarios.

Abstract

In this paper, a progress recognition method consider occlusion using deep metric learning is proposed to visualize the product assembly process in a factory. First, the target assembly product is detected from images acquired from a fixed-point camera installed in the factory using a deep learning-based object detection method. Next, the detection area is cropped from the image. Finally, by using a classification method based on deep metric learning on the cropped image, the progress of the product assembly work is estimated as a rough progress step. As a specific progress estimation model, we propose an Anomaly Triplet-Net that adds anomaly samples to Triplet Loss for progress estimation considering occlusion. In experiments, an 82.9% success rate is achieved for the progress estimation method using Anomaly Triplet-Net. We also experimented with the practicality of the sequence of detection, cropping, and progression estimation, and confirmed the effectiveness of the overall system.