IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly

TL;DR

IMPACT is a comprehensive, multi-view RGB-D dataset for industrial assembly tasks, enabling advanced procedural understanding and anomaly detection in real-world scenarios.

Contribution

First industrial assembly benchmark combining synchronized multi-view RGB-D data, detailed annotations, and anomaly-recovery supervision for procedural understanding.

Findings

Baseline models reveal limitations under realistic conditions.

The dataset captures diverse assembly and disassembly workflows.

Annotations enable detailed analysis of operator actions and compliance.

Abstract

We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly and disassembly of a commercial angle grinder with professional-grade tools. To our knowledge, IMPACT is the first real industrial assembly benchmark that jointly provides synchronized ego-exo RGB-D capture, decoupled bimanual annotation, compliance-aware state tracking, and explicit anomaly--recovery supervision within a single real industrial workflow. It comprises 112 trials from 13 participants totaling 39.5 hours, with multi-route execution governed by a partial-order prerequisite graph, a six-category anomaly taxonomy, and operator cognitive load measured via NASA-TLX. The annotation hierarchy links hand-specific atomic actions to coarse procedural steps, component assembly states, and per-hand compliance phases, with synchronized null spans across views to decouple perceptual limitations from algorithmic failure. Systematic baselines reveal fundamental limitations that remain invisible to single-task benchmarks, particularly under realistic deployment conditions that involve incomplete observations, flexible execution paths, and corrective behavior. The full dataset, annotations, and evaluation code are available at https://github.com/Kratos-Wen/IMPACT.