Privacy-Aware Activity Classification from First Person Office Videos

TL;DR

This paper presents a privacy-aware system for classifying human activities in office videos captured by first-person cameras, using object blurring and ensemble neural networks, achieving high accuracy while protecting sensitive information.

Contribution

It introduces a novel privacy-preserving activity classification system combining object detection, blurring, and ensemble RNNs, evaluated on a public dataset with competitive results.

Findings

Achieved 85.08% accuracy on unprotected videos.

Maintained 73.68% accuracy on privacy-protected videos.

System won 3rd prize in IEEE VIP Cup 2019.

Abstract

In the advent of wearable body-cameras, human activity classification from First-Person Videos (FPV) has become a topic of increasing importance for various applications, including in life-logging, law-enforcement, sports, workplace, and healthcare. One of the challenging aspects of FPV is its exposure to potentially sensitive objects within the user's field of view. In this work, we developed a privacy-aware activity classification system focusing on office videos. We utilized a Mask-RCNN with an Inception-ResNet hybrid as a feature extractor for detecting, and then blurring out sensitive objects (e.g., digital screens, human face, paper) from the videos. For activity classification, we incorporate an ensemble of Recurrent Neural Networks (RNNs) with ResNet, ResNext, and DenseNet based feature extractors. The proposed system was trained and evaluated on the FPV office video dataset that includes 18-classes made available through the IEEE Video and Image Processing (VIP) Cup 2019 competition. On the original unprotected FPVs, the proposed activity classifier ensemble reached an accuracy of 85.078% with precision, recall, and F1 scores of 0.88, 0.85 & 0.86, respectively. On privacy protected videos, the performances were slightly degraded, with accuracy, precision, recall, and F1 scores at 73.68%, 0.79, 0.75, and 0.74, respectively. The presented system won the 3rd prize in the IEEE VIP Cup 2019 competition.