Boosting Adverse Weather Crowd Counting via Multi-queue Contrastive Learning

TL;DR

This paper introduces Multi-queue Contrastive Learning (MQCL), a two-stage method that enhances crowd counting accuracy under adverse weather by learning weather-aware representations and converting them to normal weather domain.

Contribution

The paper proposes a novel two-stage framework with multi-queue contrastive learning to address weather domain gaps and class imbalance in crowd counting.

Findings

Reduces counting error under adverse weather by 22%.

Achieves state-of-the-art performance with minimal computational increase.

Effectively learns weather-aware representations and domain conversion.

Abstract

Currently, most crowd counting methods have outstanding performance under normal weather conditions. However, our experimental validation reveals two key obstacles limiting the accuracy improvement of crowd counting models: 1) the domain gap between the adverse weather and the normal weather images; 2) the weather class imbalance in the training set. To address the problems, we propose a two-stage crowd counting method named Multi-queue Contrastive Learning (MQCL). Specifically, in the first stage, our target is to equip the backbone network with weather-awareness capabilities. In this process, a contrastive learning method named multi-queue MoCo designed by us is employed to enable representation learning under weather class imbalance. After the first stage is completed, the backbone model is "mature" enough to extract weather-related representations. On this basis, we proceed to the second stage, in which we propose to refine the representations under the guidance of contrastive learning, enabling the conversion of the weather-aware representations to the normal weather domain. Through such representation and conversion, the model achieves robust counting performance under both normal and adverse weather conditions. Extensive experimental results show that, compared to the baseline, MQCL reduces the counting error under adverse weather conditions by 22%, while introducing only about 13% increase in computational burden, which achieves state-of-the-art performance.