hear-your-action: human action recognition by ultrasound active sensing

TL;DR

This paper introduces a privacy-preserving human action recognition method using ultrasound active sensing, achieving high accuracy across different individuals and environments, and creating a new dataset for this purpose.

Contribution

It proposes a novel ultrasound-based action recognition approach, including a new dataset and feature comparison, addressing privacy concerns in visual-based methods.

Findings

Achieved 97.9% accuracy within the same person and environment.

Achieved 89.5% accuracy across different people.

Demonstrated robustness of ultrasound sensing for action recognition.

Abstract

Action recognition is a key technology for many industrial applications. Methods using visual information such as images are very popular. However, privacy issues prevent widespread usage due to the inclusion of private information, such as visible faces and scene backgrounds, which are not necessary for recognizing user action. In this paper, we propose a privacy-preserving action recognition by ultrasound active sensing. As action recognition from ultrasound active sensing in a non-invasive manner is not well investigated, we create a new dataset for action recognition and conduct a comparison of features for classification. We calculated feature values by focusing on the temporal variation of the amplitude of ultrasound reflected waves and performed classification using a support vector machine and VGG for eight fundamental action classes. We confirmed that our method achieved an accuracy of 97.9% when trained and evaluated on the same person and in the same environment. Additionally, our method achieved an accuracy of 89.5% even when trained and evaluated on different people. We also report the analyses of accuracies in various conditions and limitations.