Batch-based Activity Recognition from Egocentric Photo-Streams Revisited

TL;DR

This paper presents a batch-based activity recognition system using egocentric photo-streams, combining CNNs and RNNs, achieving high accuracy and outperforming existing methods in classifying daily activities.

Contribution

It introduces a novel batch-based approach that integrates CNNs and RNNs for activity recognition from photo-streams without relying on event boundaries.

Findings

Achieved 89.85% accuracy on a large egocentric photo dataset.

Outperformed state-of-the-art end-to-end methods.

Effectively models temporal evolution of activities.

Abstract

Wearable cameras can gather large a\-mounts of image data that provide rich visual information about the daily activities of the wearer. Motivated by the large number of health applications that could be enabled by the automatic recognition of daily activities, such as lifestyle characterization for habit improvement, context-aware personal assistance and tele-rehabilitation services, we propose a system to classify 21 daily activities from photo-streams acquired by a wearable photo-camera. Our approach combines the advantages of a Late Fusion Ensemble strategy relying on convolutional neural networks at image level with the ability of recurrent neural networks to account for the temporal evolution of high level features in photo-streams without relying on event boundaries. The proposed batch-based approach achieved an overall accuracy of 89.85\%, outperforming state of the art end-to-end methodologies. These results were achieved on a dataset consists of 44,902 egocentric pictures from three persons captured during 26 days in average.