Online recognition of unsegmented actions with hierarchical SOM architecture

TL;DR

This paper introduces a hierarchical self-organizing neural network architecture for real-time recognition of unsegmented actions, effectively handling segmentation and classification in online scenarios with promising results.

Contribution

It presents a novel three-layer neural architecture using self-organizing maps for online unsegmented action recognition, combining segmentation and classification.

Findings

System performs well in online experiments

Slight performance drop compared to offline tests

Architecture is practical for real-world applications

Abstract

Automatic recognition of an online series of unsegmented actions requires a method for segmentation that determines when an action starts and when it ends. In this paper, a novel approach for recognizing unsegmented actions in online test experiments is proposed. The method uses self-organizing neural networks to build a three-layer cognitive architecture. The unique features of an action sequence are represented as a series of elicited key activations by the first-layer self-organizing map. An average length of a key activation vector is calculated for all action sequences in a training set and adjusted in learning trials to generate input patterns to the second-layer self-organizing map. The pattern vectors are clustered in the second layer, and the clusters are then labeled by an action identity in the third layer neural network. The experiment results show that although the performance drops slightly in online experiments compared to the offline tests, the ability of the proposed architecture to deal with the unsegmented action sequences as well as the online performance makes the system more plausible and practical in real-case scenarios.