CrowdCam: Dynamic Region Segmentation

TL;DR

This paper introduces a new algorithm for segmenting dynamic regions in CrowdCam images, combining geometric, appearance, and proximity cues, and demonstrates significant improvements over existing methods on a new dataset.

Contribution

The paper presents a comprehensive, multi-cue approach tailored for CrowdCam images, addressing the unique challenges of sparse, multi-view dynamic scene segmentation.

Findings

Nearly doubles the success score compared to state-of-the-art methods.

Effective integration of geometry, appearance, and proximity cues.

Validated on a newly collected, challenging dataset.

Abstract

We consider the problem of segmenting dynamic regions in CrowdCam images, where a dynamic region is the projection of a moving 3D object on the image plane. Quite often, these regions are the most interesting parts of an image. CrowdCam images is a set of images of the same dynamic event, captured by a group of non-collaborating users. Almost every event of interest today is captured this way. This new type of images raises the need to develop new algorithms tailored specifically for it. We propose a comprehensive solution to the problem. Our solution combines cues that are based on geometry, appearance and proximity. First, geometric reasoning is used to produce rough score maps that determine, for every pixel, how likely it is to be the projection of a static or dynamic scene point. These maps are noisy because CrowdCam images are usually few and far apart both in space and in time. Then, we use similarity in appearance space and proximity in the image plane to encourage neighboring pixels to be labeled similarly as either static or dynamic. We collected a new, and challenging, data set to evaluate our algorithm. Results show that the success score of our algorithm is nearly double that of the current state of the art approach.