# Progress Estimation and Phase Detection for Sequential Processes

## TL;DR

This paper presents a real-time, sensor-based deep learning system for modeling, recognizing, and estimating the progress of sequential processes, enabling phase detection and remaining time prediction in applications like medical and sports events.

## Contribution

It introduces a novel multimodal deep learning architecture with a new activation function and loss, specifically designed for process progress estimation and phase detection.

## Key findings

- Achieved over 86% phase detection accuracy in trauma resuscitation.
- Attained 88% accuracy in Olympic swimming phase detection.
- Reduced process completeness estimation error to below 12.6%.

## Abstract

Process modeling and understanding are fundamental for advanced human-computer interfaces and automation systems. Most recent research has focused on activity recognition, but little has been done on sensor-based detection of process progress. We introduce a real-time, sensor-based system for modeling, recognizing and estimating the progress of a work process. We implemented a multimodal deep learning structure to extract the relevant spatio-temporal features from multiple sensory inputs and used a novel deep regression structure for overall completeness estimation. Using process completeness estimation with a Gaussian mixture model, our system can predict the phase for sequential processes. The performance speed, calculated using completeness estimation, allows online estimation of the remaining time. To train our system, we introduced a novel rectified hyperbolic tangent (rtanh) activation function and conditional loss. Our system was tested on data obtained from the medical process (trauma resuscitation) and sports events (Olympic swimming competition). Our system outperformed the existing trauma-resuscitation phase detectors with a phase detection accuracy of over 86%, an F1-score of 0.67, a completeness estimation error of under 12.6%, and a remaining-time estimation error of less than 7.5 minutes. For the Olympic swimming dataset, our system achieved an accuracy of 88%, an F1-score of 0.58, a completeness estimation error of 6.3% and a remaining-time estimation error of 2.9 minutes.