Robust Assembly Progress Estimation via Deep Metric Learning

TL;DR

This paper presents a robust deep metric learning system for assembly progress estimation that effectively handles subtle visual changes and occlusions, improving accuracy in manual assembly monitoring within smart factories.

Contribution

It introduces a Quadruplet Loss-based learning approach and a custom data loader to enhance assembly progress estimation accuracy, especially in challenging visual scenarios.

Findings

Improved estimation accuracy by 1.3% over existing methods.

Reduced misclassification between adjacent tasks by 1.9%.

Effective in scenarios with occlusion and minimal visual change.

Abstract

In recent years, the advancement of AI technologies has accelerated the development of smart factories. In particular, the automatic monitoring of product assembly progress is crucial for improving operational efficiency, minimizing the cost of discarded parts, and maximizing factory productivity. However, in cases where assembly tasks are performed manually over multiple days, implementing smart factory systems remains a challenge. Previous work has proposed Anomaly Triplet-Net, which estimates assembly progress by applying deep metric learning to the visual features of products. Nevertheless, when visual changes between consecutive tasks are subtle, misclassification often occurs. To address this issue, this paper proposes a robust system for estimating assembly progress, even in cases of occlusion or minimal visual change, using a small-scale dataset. Our method leverages a Quadruplet Loss-based learning approach for anomaly images and introduces a custom data loader that strategically selects training samples to enhance estimation accuracy. We evaluated our approach using a image datasets: captured during desktop PC assembly. The proposed Anomaly Quadruplet-Net outperformed existing methods on the dataset. Specifically, it improved the estimation accuracy by 1.3% and reduced misclassification between adjacent tasks by 1.9% in the desktop PC dataset and demonstrating the effectiveness of the proposed method.