Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining

TL;DR

This paper introduces a reference architecture that integrates computer vision with process mining to extract process events from unstructured video data, addressing blind spots in digitized process analysis.

Contribution

It presents a novel reference architecture and prototype for automatically deriving process events from video data, bridging the gap between computer vision and process mining.

Findings

Prototype successfully extracts process-relevant events from videos

Architecture enables flexible, use-case-specific instantiations

Evaluation confirms effectiveness in real-world scenarios

Abstract

Process mining is one of the most active research streams in business process management. In recent years, numerous methods have been proposed for analyzing structured process data. Yet, in many cases, it is only the digitized parts of processes that are directly captured from process-aware information systems, and manual activities often result in blind spots. While the use of video cameras to observe these activities could help to fill this gap, a standardized approach to extracting event logs from unstructured video data remains lacking. Here, we propose a reference architecture to bridge the gap between computer vision and process mining. Various evaluation activities (i.e., competing artifact analysis, prototyping, and real-world application) ensured that the proposed reference architecture allows flexible, use-case-driven, and context-specific instantiations. Our results also show that an exemplary software prototype instantiation of the proposed reference architecture is capable of automatically extracting most of the process-relevant events from unstructured video data.