Open-World Object Counting in Videos

TL;DR

This paper presents CountVid, a new model for open-world object counting in videos, capable of counting specific objects based on text or image prompts, with a new dataset called VideoCount for evaluation.

Contribution

The paper introduces CountVid, a novel model for open-world object counting in videos, and provides the VideoCount dataset for benchmarking this task.

Findings

CountVid achieves high accuracy in object counting.

It significantly outperforms existing baselines.

The VideoCount dataset enables robust evaluation.

Abstract

We introduce a new task of open-world object counting in videos: given a text description, or an image example, that specifies the target object, the objective is to enumerate all the unique instances of the target objects in the video. This task is especially challenging in crowded scenes with occlusions and objects of similar appearance, where avoiding double counting and identifying reappearances is crucial. To this end, we make the following contributions: we introduce a model, CountVid, for this task. It leverages an image-based counting model, and a promptable video segmentation and tracking model, to enable automated open-world object counting across video frames. To evaluate its performance, we introduce VideoCount, a new dataset for this novel task built from the TAO and MOT20 tracking datasets, as well as from videos of penguins and metal alloy crystallization captured by x-rays. Using this dataset, we demonstrate that CountVid provides accurate object counts, and significantly outperforms strong baselines. The VideoCount dataset, the CountVid model, and all the code are available at https://www.robots.ox.ac.uk/~vgg/research/countvid/.