RACANet: Reliability-Aware Crowd Anchor Network for RGB-T Crowd Counting

TL;DR

RACANet is a novel RGB-T crowd counting framework that explicitly models local spatial discrepancies and modality reliability, leading to improved accuracy and interpretability in complex scenes.

Contribution

The paper introduces a two-stage fusion framework with explicit local alignment and reliability modeling, enhancing cross-modal fusion for crowd counting.

Findings

RACANet outperforms existing methods on benchmark datasets.

The proposed local anchor fusion improves feature aggregation.

Reliability-aware modeling enhances counting accuracy in complex scenes.

Abstract

RGB-Thermal (T) crowd counting aims to integrate visible-spectrum and thermal infrared information to improve the robustness of crowd density estimation in complex scenes. Although existing studies generally improve counting accuracy through cross-modal feature fusion, most current methods rely on implicit cross-modal fusion strategies and lack explicit modeling of local spatial discrepancies as well as fine-grained characterization of modality reliability at the positional level, thereby limiting the accuracy and interpretability of the fusion process. To address these issues, this paper proposes a two-stage fusion framework, RACANet, a Reliability-Aware Crowd Anchor Network for RGB-T crowd counting. First, we introduce a lightweight cross-modal alignment pretraining stage, which explicitly learns cross-modal semantic correspondences through crowd-prior supervision and local bidirectional soft matching. Then, based on the priors learned during pretraining, a Local Anchor Fusion Module (LAFM) is introduced in the formal training stage. This module generates local semantic anchors by aggregating features from highly reliable regions and further enables adaptive pixel-level feature redistribution with a local attention mechanism. In addition, we propose a discrepancy-aware consistency constraint to dynamically coordinate the reliability of regions where modal representations are consistent. Experiments conducted on two widely used benchmark datasets, RGBT-CC and Drone-RGBT, demonstrate that RACANet outperforms existing methods. The anonymous code is available at https://anonymous.4open.science/r/RACANet-9985.