Supervised Representation Learning towards Generalizable Assembly State Recognition

TL;DR

This paper introduces a representation learning method with a novel loss function for assembly state recognition, improving robustness and scalability in identifying correct and erroneous assembly states from images.

Contribution

It proposes the intermediate-state informed loss (ISIL) that leverages unlabeled transition data, enhancing clustering and classification performance in assembly state recognition.

Findings

Significant improvements in clustering and classification across architectures.

Accurate distinction between correct and error states.

Effective recognition despite training on error-free images.

Abstract

Assembly state recognition facilitates the execution of assembly procedures, offering feedback to enhance efficiency and minimize errors. However, recognizing assembly states poses challenges in scalability, since parts are frequently updated, and the robustness to execution errors remains underexplored. To address these challenges, this paper proposes an approach based on representation learning and the novel intermediate-state informed loss function modification (ISIL). ISIL leverages unlabeled transitions between states and demonstrates significant improvements in clustering and classification performance for all tested architectures and losses. Despite being trained exclusively on images without execution errors, thorough analysis on error states demonstrates that our approach accurately distinguishes between correct states and states with various types of execution errors. The integration of the proposed algorithm can offer meaningful assistance to workers and mitigate unexpected losses due to procedural mishaps in industrial settings. The code is available at: https://timschoonbeek.github.io/state_rec