PLSM: A Parallelized Liquid State Machine for Unintentional Action Detection

TL;DR

This paper introduces PLSM, a novel parallelized Liquid State Machine architecture optimized for low-end hardware, capable of classifying unintentional actions in videos more efficiently than traditional deep learning models.

Contribution

The paper presents the first formulation of a spatio-temporal read-out layer with semantic constraints in LSMs, along with a GPU-compatible parallel implementation for unintentional action detection.

Findings

PLSM outperforms self-supervised models in unintentional action detection.

PLSM is computationally lighter than traditional deep learning models.

The approach is suitable for deployment on low-end embedded systems.

Abstract

Reservoir Computing (RC) offers a viable option to deploy AI algorithms on low-end embedded system platforms. Liquid State Machine (LSM) is a bio-inspired RC model that mimics the cortical microcircuits and uses spiking neural networks (SNN) that can be directly realized on neuromorphic hardware. In this paper, we present a novel Parallelized LSM (PLSM) architecture that incorporates spatio-temporal read-out layer and semantic constraints on model output. To the best of our knowledge, such a formulation has been done for the first time in literature, and it offers a computationally lighter alternative to traditional deep-learning models. Additionally, we also present a comprehensive algorithm for the implementation of parallelizable SNNs and LSMs that are GPU-compatible. We implement the PLSM model to classify unintentional/accidental video clips, using the Oops dataset. From the experimental results on detecting unintentional action in video, it can be observed that our proposed model outperforms a self-supervised model and a fully supervised traditional deep learning model. All the implemented codes can be found at our repository https://github.com/anonymoussentience2020/Parallelized_LSM_for_Unintentional_Action_Recognition.