In-Sensor Motion Recognition with Memristive System and Light Sensing Surfaces

TL;DR

This paper presents an innovative in-sensor motion recognition system combining memristive devices and light-sensing surfaces, achieving high accuracy and ultra-low energy consumption for edge applications.

Contribution

The work introduces a new device architecture integrating memristive systems with light sensors for energy-efficient, in-sensor motion classification at the edge.

Findings

Achieves 97.22% accuracy in gesture recognition.

Requires only 4.17 nJ for processing and 0.952 nJ for testing per movement.

Operates effectively under 5% noise interference.

Abstract

In this paper, we introduce a novel device architecture that merges memristive devices with light-sensing surfaces, for energy-efficient motion recognition at the edge. Our light-sensing surface captures motion data through in-sensor computation. This data is then processed using a memristive system equipped with a HfO2-based synaptic device, coupled with a winner-take-all (WTA) circuit, tailored for low-power motion classification tasks. We validate our end-to-end system using four distinct human hand gestures - left-to-right, right-to-left, bottom-to-top, and top-to-bottom movements - to assess energy efficiency and classification robustness. Our experiments show that the system requires an average of only 4.17 nJ for taking our processed analog signal and mapping weights onto our memristive system and 0.952 nJ for testing per movement class, achieving 97.22% accuracy even under 5% noise interference. A key advantage of our proposed architecture is its low energy requirement, enabling the integration of energy-harvesting solutions such as solar power for sustainable autonomous operation. Additionally, our approach enhances data privacy by processing data locally, reducing the need for external data transmission and storage.