Behavioural pattern discovery from collections of egocentric photo-streams

TL;DR

This paper introduces a novel unsupervised semantic clustering method to identify individual behavioral patterns from egocentric photo-streams, validated on real-world data from multiple users.

Contribution

It presents a new approach combining context-based time-frame characterization and semantic clustering to discover personal behavioral routines from egocentric images.

Findings

Behavioral patterns can be effectively extracted from egocentric photo-streams.

The method accurately characterizes individual routines and lifestyles.

Validation on 104 days and over 100,000 images demonstrates robustness.

Abstract

The automatic discovery of behaviour is of high importance when aiming to assess and improve the quality of life of people. Egocentric images offer a rich and objective description of the daily life of the camera wearer. This work proposes a new method to identify a person's patterns of behaviour from collected egocentric photo-streams. Our model characterizes time-frames based on the context (place, activities and environment objects) that define the images composition. Based on the similarity among the time-frames that describe the collected days for a user, we propose a new unsupervised greedy method to discover the behavioural pattern set based on a novel semantic clustering approach. Moreover, we present a new score metric to evaluate the performance of the proposed algorithm. We validate our method on 104 days and more than 100k images extracted from 7 users. Results show that behavioural patterns can be discovered to characterize the routine of individuals and consequently their lifestyle.