Enabling High-Accuracy Human Activity Recognition with Fine-Grained Indoor Localization

TL;DR

This paper presents LEHAR, a smartphone-based human activity recognition system that combines acceleration, audio, and Wi-Fi localization data to achieve high accuracy without extensive sensor deployment.

Contribution

The study introduces a novel LEHAR system that leverages indoor localization with Wi-Fi RTT to enhance HAR accuracy using only a smartphone, outperforming existing methods.

Findings

LEHAR achieved an F1-score of 0.965 on 12 activities.

Existing methods scored 0.660 and 0.865 on the same activities.

LEHAR maintained high accuracy as the number of activities increased.

Abstract

While computers play an increasingly important role in every aspect of our lives, their inability to understand what tasks users are physically performing makes a wide range of applications, including health monitoring and context-specific assistance, difficult or impossible. With Human Activity Recognition (HAR), applications could track if a patient took his pills and detect the behavioral changes associated with diseases such as Alzheimer's. Current systems for HAR require diverse sensors (e.g., cameras, microphones, proximity sensors, and accelerometers) placed throughout the environment to provide detailed observations needed for high-accuracy HAR. The difficulty of instrumenting an environment with these sensors makes this approach impractical. This project considers whether recent advances in indoor localization (Wi-Fi Round Trip Time) enable high-accuracy HAR using only a smartphone. My design, called Location-Enhanced HAR (LEHAR), uses machine learning to combine acceleration, audio, and location data to detect common human activities. A LEHAR prototype, designed to recognize a dozen common activities conducted in a typical household, achieved an F1-score of 0.965. In contrast, existing approaches, which use only acceleration or audio data, obtained F1-scores of 0.660 and 0.865, respectively, on the same activities. In addition, the F1-score of existing designs dropped significantly as more activities were added for recognition, while LEHAR was able to maintain high accuracy. The results show that using a combination of acceleration, audio, and Wi-Fi Round Trip Time localization can enable a highly accurate and easily deployable HAR system.