Co-interest Person Detection from Multiple Wearable Camera Videos

TL;DR

This paper introduces a novel method for detecting the co-interest person in videos from multiple wearable cameras by leveraging motion patterns rather than appearance, enabling identification even among similar-looking individuals.

Contribution

The paper proposes a motion-based approach using CRF modeling to locate the co-interest person across synchronized wearable camera videos, addressing appearance similarity issues.

Findings

Effective detection of co-interest persons in challenging scenarios

Robustness to appearance similarities among individuals

Validated on three real-world wearable camera datasets

Abstract

Wearable cameras, such as Google Glass and Go Pro, enable video data collection over larger areas and from different views. In this paper, we tackle a new problem of locating the co-interest person (CIP), i.e., the one who draws attention from most camera wearers, from temporally synchronized videos taken by multiple wearable cameras. Our basic idea is to exploit the motion patterns of people and use them to correlate the persons across different videos, instead of performing appearance-based matching as in traditional video co-segmentation/localization. This way, we can identify CIP even if a group of people with similar appearance are present in the view. More specifically, we detect a set of persons on each frame as the candidates of the CIP and then build a Conditional Random Field (CRF) model to select the one with consistent motion patterns in different videos and high spacial-temporal consistency in each video. We collect three sets of wearable-camera videos for testing the proposed algorithm. All the involved people have similar appearances in the collected videos and the experiments demonstrate the effectiveness of the proposed algorithm.