Dynamic Vision Sensors for Human Activity Recognition

TL;DR

This paper investigates the use of Dynamic Vision Sensors (DVS) for human activity recognition, demonstrating that DVS can achieve comparable performance to conventional video despite capturing sparse motion data.

Contribution

The study introduces a novel approach of using DVS motion maps combined with MBH for human activity recognition, showing promising results on benchmark and real datasets.

Findings

DVS-based HAR achieves performance comparable to traditional video-based methods.

Fusing motion maps with MBH improves recognition accuracy.

DVS offers low power and high dynamic range advantages for wearable applications.

Abstract

Unlike conventional cameras which capture video at a fixed frame rate, Dynamic Vision Sensors (DVS) record only changes in pixel intensity values. The output of DVS is simply a stream of discrete ON/OFF events based on the polarity of change in its pixel values. DVS has many attractive features such as low power consumption, high temporal resolution, high dynamic range and fewer storage requirements. All these make DVS a very promising camera for potential applications in wearable platforms where power consumption is a major concern. In this paper, we explore the feasibility of using DVS for Human Activity Recognition (HAR). We propose to use the various slices (such as $x-y$, $x-t$, and $y-t$) of the DVS video as a feature map for HAR and denote them as Motion Maps. We show that fusing motion maps with Motion Boundary Histogram (MBH) give good performance on the benchmark DVS dataset as well as on a real DVS gesture dataset collected by us. Interestingly, the performance of DVS is comparable to that of conventional videos although DVS captures only sparse motion information.