An Internal Clock Based Space-time Neural Network for Motion Speed Recognition

TL;DR

This paper introduces a novel space-time neural network with an internal clock mechanism for recognizing human motion speeds, demonstrating high accuracy with minimal training data and low power consumption, suitable for edge AI applications.

Contribution

The work presents a new internal clock-based SNN architecture that automatically tunes its frequency for motion speed recognition, requiring only small datasets and offering fast, low-power learning.

Findings

Achieves up to 83.3% accuracy on cartoon videos

Recognizes subtle speed differences like run vs. fast walk

Requires only six datasets and 42 training trials

Abstract

In this work we present a novel internal clock based space-time neural network for motion speed recognition. The developed system has a spike train encoder, a Spiking Neural Network (SNN) with internal clocking behaviors, a pattern transformation block and a Network Dynamic Dependent Plasticity (NDDP) learning block. The core principle is that the developed SNN will automatically tune its network pattern frequency (internal clock frequency) to recognize human motions in a speed domain. We employed both cartoons and real-world videos as training benchmarks, results demonstrate that our system can not only recognize motions with considerable speed differences (e.g. run, walk, jump, wonder(think) and standstill), but also motions with subtle speed gaps such as run and fast walk. The inference accuracy can be up to 83.3% (cartoon videos) and 75% (real-world videos). Meanwhile, the system only requires six video datasets in the learning stage and with up to 42 training trials. Hardware performance estimation indicates that the training time is 0.84-4.35s and power consumption is 33.26-201mW (based on an ARM Cortex M4 processor). Therefore, our system takes unique learning advantages of the requirement of the small dataset, quick learning and low power performance, which shows great potentials for edge or scalable AI-based applications.