Social Style Characterization from Egocentric Photo-streams

TL;DR

This paper introduces a system that automatically detects, categorizes, and analyzes social interactions from egocentric photo-streams using machine learning, providing insights into social patterns over time.

Contribution

It presents a novel pipeline combining event detection, social interaction classification, and clustering to characterize social patterns from wearable photo-stream data.

Findings

Promising accuracy in detecting social interactions

Effective categorization into formal and informal events

Successful clustering of social relations over time

Abstract

This paper proposes a system for automatic social pattern characterization using a wearable photo-camera. The proposed pipeline consists of three major steps. First, detection of people with whom the camera wearer interacts and, second, categorization of the detected social interactions into formal and informal. These two steps act at event-level where each potential social event is modeled as a multi-dimensional time-series, whose dimensions correspond to a set of relevant features for each task, and a LSTM network is employed for time-series classification. In the last step, recurrences of the same person across the whole set of social interactions are clustered to achieve a comprehensive understanding of the diversity and frequency of the social relations of the user. Experiments over a dataset acquired by a user wearing a photo-camera during a month show promising results on the task of social pattern characterization from egocentric photo-streams.