R-Clustering for Egocentric Video Segmentation

TL;DR

This paper introduces R-Clustering, a novel egocentric video segmentation method combining statistical change detection and agglomerative clustering within an energy framework, improving over-segmentation issues.

Contribution

The paper proposes an integrated approach that combines clustering with concept drift detection in an energy-minimization framework for better video segmentation.

Findings

Outperforms state-of-the-art clustering methods on egocentric videos

Effective in handling oversegmentation issues

Validated on over 13,000 images from wearable cameras

Abstract

In this paper, we present a new method for egocentric video temporal segmentation based on integrating a statistical mean change detector and agglomerative clustering(AC) within an energy-minimization framework. Given the tendency of most AC methods to oversegment video sequences when clustering their frames, we combine the clustering with a concept drift detection technique (ADWIN) that has rigorous guarantee of performances. ADWIN serves as a statistical upper bound for the clustering-based video segmentation. We integrate both techniques in an energy-minimization framework that serves to disambiguate the decision of both techniques and to complete the segmentation taking into account the temporal continuity of video frames descriptors. We present experiments over egocentric sets of more than 13.000 images acquired with different wearable cameras, showing that our method outperforms state-of-the-art clustering methods.