Preliminary Investigation of SSL for Complex Work Activity Recognition in Industrial Domain via MoIL

TL;DR

This paper introduces MoIL, a self-supervised learning approach that improves complex work activity recognition from wearable sensor data by focusing on characteristic action motifs, achieving state-of-the-art results with limited labeled data.

Contribution

The study presents a novel SSL method called Motif Identification Learning (MoIL) that effectively captures characteristic actions for activity recognition in complex industrial tasks.

Findings

MoIL outperforms existing methods on real-world data.

It enables accurate activity recognition with limited labeled data.

State-of-the-art performance achieved in complex work scenarios.

Abstract

In this study, we investigate a new self-supervised learning (SSL) approach for complex work activity recognition using wearable sensors. Owing to the cost of labeled sensor data collection, SSL methods for human activity recognition (HAR) that effectively use unlabeled data for pretraining have attracted attention. However, applying prior SSL to complex work activities such as packaging works is challenging because the observed data vary considerably depending on situations such as the number of items to pack and the size of the items in the case of packaging works. In this study, we focus on sensor data corresponding to characteristic and necessary actions (sensor data motifs) in a specific activity such as a stretching packing tape action in an assembling a box activity, and \textcolor{black}{try} to train a neural network in self-supervised learning so that it identifies occurrences of the characteristic actions, i.e., Motif Identification Learning (MoIL). The feature extractor in the network is used in the downstream task, i.e., work activity recognition, enabling precise activity recognition containing characteristic actions with limited labeled training data. The MoIL approach was evaluated on real-world work activity data and it achieved state-of-the-art performance under limited training labels.