Predicting the intended action using internal simulation of perception

TL;DR

This paper introduces a hierarchical neural architecture that predicts human actions by internally simulating perceptual states, enhancing recognition accuracy and addressing sensory limitations.

Contribution

It presents a novel use of associative self-organising neural networks for action prediction through internal perception simulation.

Findings

Improved action recognition performance across three datasets.

Internal simulation helps predict future perceptual sequences.

Addresses sensory input limitations effectively.

Abstract

This article proposes an architecture, which allows the prediction of intention by internally simulating perceptual states represented by action pattern vectors. To this end, associative self-organising neural networks (A-SOM) is utilised to build a hierarchical cognitive architecture for recognition and simulation of the skeleton based human actions. The abilities of the proposed architecture in recognising and predicting actions is evaluated in experiments using three different datasets of 3D actions. Based on the experiments of this article, applying internally simulated perceptual states represented by action pattern vectors improves the performance of the recognition task in all experiments. Furthermore, internal simulation of perception addresses the problem of having limited access to the sensory input, and also the future prediction of the consecutive perceptual sequences. The performance of the system is compared and discussed with similar architecture using self-organizing neural networks (SOM).