Low-Power On-Device Gesture Recognition with Einsum Networks

TL;DR

This paper presents a low-power, on-device gesture recognition system using Einsum Networks, enabling efficient, explainable, and accurate gesture detection on resource-constrained wearable devices.

Contribution

It introduces a novel gesture recognition pipeline leveraging Einsum Networks for energy-efficient inference on distributed, resource-limited devices.

Findings

Outperforms benchmark models in gesture recognition accuracy

Demonstrates energy efficiency suitable for low-power devices

Validates system in real-world RF-based gesture scenarios

Abstract

We design a gesture-recognition pipeline for networks of distributed, resource constrained devices utilising Einsum Networks. Einsum Networks are probabilistic circuits that feature a tractable inference, explainability, and energy efficiency. The system is validated in a scenario of low-power, body-worn, passive Radio Frequency Identification-based gesture recognition. Each constrained device includes task-specific processing units responsible for Received Signal Strength (RSS) and phase processing or Angle of Arrival (AoA) estimation, along with feature extraction, as well as dedicated Einsum hardware that processes the extracted features. The output of all constrained devices is then fused in a decision aggregation module to predict gestures. Experimental results demonstrate that the method outperforms the benchmark models.