Multi-Face Tracking by Extended Bag-of-Tracklets in Egocentric Videos

TL;DR

This paper introduces a robust multi-face tracking method for low frame-rate egocentric videos from wearable cameras, addressing challenges like abrupt view changes and occlusions to improve social event detection.

Contribution

The novel extended bag-of-tracklets (eBoT) approach effectively tracks multiple faces in challenging egocentric videos, outperforming existing methods in robustness and accuracy.

Findings

eBoT outperforms state-of-the-art methods in accuracy

The approach handles abrupt view changes and occlusions effectively

Validated on an extensive egocentric dataset

Abstract

Wearable cameras offer a hands-free way to record egocentric images of daily experiences, where social events are of special interest. The first step towards detection of social events is to track the appearance of multiple persons involved in it. In this paper, we propose a novel method to find correspondences of multiple faces in low temporal resolution egocentric videos acquired through a wearable camera. This kind of photo-stream imposes additional challenges to the multi-tracking problem with respect to conventional videos. Due to the free motion of the camera and to its low temporal resolution, abrupt changes in the field of view, in illumination condition and in the target location are highly frequent. To overcome such difficulties, we propose a multi-face tracking method that generates a set of tracklets through finding correspondences along the whole sequence for each detected face and takes advantage of the tracklets redundancy to deal with unreliable ones. Similar tracklets are grouped into the so called extended bag-of-tracklets (eBoT), which is aimed to correspond to a specific person. Finally, a prototype tracklet is extracted for each eBoT, where the occurred occlusions are estimated by relying on a new measure of confidence. We validated our approach over an extensive dataset of egocentric photo-streams and compared it to state of the art methods, demonstrating its effectiveness and robustness.