AI-driven visual monitoring of industrial assembly tasks

TL;DR

This paper presents ViMAT, an AI-driven system for real-time visual monitoring of industrial assembly tasks that operates without the need for workspace constraints or visual markers, enhancing flexibility and applicability.

Contribution

The paper introduces ViMAT, a novel AI system combining perception and reasoning modules for flexible, markerless, real-time monitoring of complex assembly tasks.

Findings

Effective in real-world scenarios with partial observations

Successfully applied to LEGO and hydraulic press assembly tasks

Demonstrates robustness and accuracy in challenging conditions

Abstract

Visual monitoring of industrial assembly tasks is critical for preventing equipment damage due to procedural errors and ensuring worker safety. Although commercial solutions exist, they typically require rigid workspace setups or the application of visual markers to simplify the problem. We introduce ViMAT, a novel AI-driven system for real-time visual monitoring of assembly tasks that operates without these constraints. ViMAT combines a perception module that extracts visual observations from multi-view video streams with a reasoning module that infers the most likely action being performed based on the observed assembly state and prior task knowledge. We validate ViMAT on two assembly tasks, involving the replacement of LEGO components and the reconfiguration of hydraulic press molds, demonstrating its effectiveness through quantitative and qualitative analysis in challenging real-world scenarios characterized by partial and uncertain visual observations. Project page: https://tev-fbk.github.io/ViMAT