Towards Egocentric Person Re-identification and Social Pattern Analysis

TL;DR

This paper presents a model for analyzing social interactions and patterns from egocentric camera images by detecting and re-identifying faces, enabling insights into user social behavior over time.

Contribution

The study introduces a novel approach to evaluate and visualize social traits from egocentric photostreams using face detection and clustering for social pattern analysis.

Findings

Social profiles can be derived from egocentric images.

Recurrent face detection reveals social behavior patterns.

Model validated over multiple weeks with different users.

Abstract

Wearable cameras capture a first-person view of the daily activities of the camera wearer, offering a visual diary of the user behaviour. Detection of the appearance of people the camera user interacts with for social interactions analysis is of high interest. Generally speaking, social events, lifestyle and health are highly correlated, but there is a lack of tools to monitor and analyse them. We consider that egocentric vision provides a tool to obtain information and understand users social interactions. We propose a model that enables us to evaluate and visualize social traits obtained by analysing social interactions appearance within egocentric photostreams. Given sets of egocentric images, we detect the appearance of faces within the days of the camera wearer, and rely on clustering algorithms to group their feature descriptors in order to re-identify persons. Recurrence of detected faces within photostreams allows us to shape an idea of the social pattern of behaviour of the user. We validated our model over several weeks recorded by different camera wearers. Our findings indicate that social profiles are potentially useful for social behaviour interpretation.