Identification of Conversation Partners from Egocentric Video

TL;DR

This paper presents a new dataset and initial baseline results for identifying conversation partners from egocentric video, aiming to improve social interaction analysis in noisy multi-talker environments.

Contribution

It introduces a novel dataset of 69 hours of egocentric video with labeled conversation partners for developing and evaluating algorithms in social interaction analysis.

Findings

Dataset of 69 hours of egocentric video created

Initial baseline results for conversation partner identification

Facilitates future research in egocentric social interaction analysis

Abstract

Communicating in noisy, multi-talker environments is challenging, especially for people with hearing impairments. Egocentric video data can potentially be used to identify a user's conversation partners, which could be used to inform selective acoustic amplification of relevant speakers. Recent introduction of datasets and tasks in computer vision enable progress towards analyzing social interactions from an egocentric perspective. Building on this, we focus on the task of identifying conversation partners from egocentric video and describe a suitable dataset. Our dataset comprises 69 hours of egocentric video of diverse multi-conversation scenarios where each individual was assigned one or more conversation partners, providing the labels for our computer vision task. This dataset enables the development and assessment of algorithms for identifying conversation partners and evaluating related approaches. Here, we describe the dataset alongside initial baseline results of this ongoing work, aiming to contribute to the exciting advancements in egocentric video analysis for social settings.