A Light-powered, Always-On, Smart Camera with Compressed Domain Gesture Detection

TL;DR

This paper presents an energy-efficient, self-powered smart camera system that recognizes gestures directly from compressed measurements, enabling always-on applications with low energy consumption and high accuracy.

Contribution

The novel system directly extracts gesture features from compressed data and is fully powered by light energy, demonstrating practical self-powered gesture recognition.

Findings

Recognizes gestures with over 80% accuracy using only 400 compressed measurements.

Consumes only 95mJ of energy per frame, enabling low-power operation.

Operates effectively with a 768x compression ratio.

Abstract

In this paper we propose an energy-efficient camera-based gesture recognition system powered by light energy for "always on" applications. Low energy consumption is achieved by directly extracting gesture features from the compressed measurements, which are the block averages and the linear combinations of the image sensor's pixel values. The gestures are recognized using a nearest-neighbour (NN) classifier followed by Dynamic Time Warping (DTW). The system has been implemented on an Analog Devices Black Fin ULP vision processor and powered by PV cells whose output is regulated by TI's DC-DC buck converter with Maximum Power Point Tracking (MPPT). Measured data reveals that with only 400 compressed measurements (768x compression ratio) per frame, the system is able to recognize key wake-up gestures with greater than 80% accuracy and only 95mJ of energy per frame. Owing to its fully self-powered operation, the proposed system can find wide applications in "always-on" vision systems such as in surveillance, robotics and consumer electronics with touch-less operation.