DR.VIC: Decomposition and Reasoning for Video Individual Counting

TL;DR

This paper introduces DR.VIC, a novel approach for counting individual pedestrians in videos by decomposing the problem into initial and new pedestrians, avoiding traditional tracking methods, and demonstrating superior performance on challenging datasets.

Contribution

The paper proposes a new video individual counting framework that decomposes pedestrians into initial and new groups, using an end-to-end network with density estimation and optimal transport reasoning.

Findings

Outperforms baseline methods on congested pedestrian datasets.

Effectively counts individual pedestrians without relying on tracking.

Demonstrates robustness across diverse crowded scenes.

Abstract

Pedestrian counting is a fundamental tool for understanding pedestrian patterns and crowd flow analysis. Existing works (e.g., image-level pedestrian counting, crossline crowd counting et al.) either only focus on the image-level counting or are constrained to the manual annotation of lines. In this work, we propose to conduct the pedestrian counting from a new perspective - Video Individual Counting (VIC), which counts the total number of individual pedestrians in the given video (a person is only counted once). Instead of relying on the Multiple Object Tracking (MOT) techniques, we propose to solve the problem by decomposing all pedestrians into the initial pedestrians who existed in the first frame and the new pedestrians with separate identities in each following frame. Then, an end-to-end Decomposition and Reasoning Network (DRNet) is designed to predict the initial pedestrian count with the density estimation method and reason the new pedestrian's count of each frame with the differentiable optimal transport. Extensive experiments are conducted on two datasets with congested pedestrians and diverse scenes, demonstrating the effectiveness of our method over baselines with great superiority in counting the individual pedestrians. Code: https://github.com/taohan10200/DRNet.