FSCA-Net: Feature-Separated Cross-Attention Network for Robust Multi-Dataset Training

TL;DR

This paper introduces FSCA-Net, a novel framework that disentangles features into domain-invariant and domain-specific parts with cross-attention and mutual information optimization, significantly improving multi-dataset crowd counting performance.

Contribution

The paper proposes a unified model with feature separation, cross-attention fusion, and mutual information objectives to enhance cross-dataset generalization in crowd counting.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively mitigates negative transfer across datasets.

Demonstrates robustness and scalability in real-world scenarios.

Abstract

Crowd counting plays a vital role in public safety, traffic regulation, and smart city management. However, despite the impressive progress achieved by CNN- and Transformer-based models, their performance often deteriorates when applied across diverse environments due to severe domain discrepancies. Direct joint training on multiple datasets, which intuitively should enhance generalization, instead results in negative transfer, as shared and domain-specific representations become entangled. To address this challenge, we propose the Feature Separation and Cross-Attention Network FSCA-Net, a unified framework that explicitly disentangles feature representations into domain-invariant and domain-specific components. A novel cross-attention fusion module adaptively models interactions between these components, ensuring effective knowledge transfer while preserving dataset-specific discriminability. Furthermore, a mutual information optimization objective is introduced to maximize consistency among domain-invariant features and minimize redundancy among domain-specific ones, promoting complementary shared-private representations. Extensive experiments on multiple crowd counting benchmarks demonstrate that FSCA-Net effectively mitigates negative transfer and achieves state-of-the-art cross-dataset generalization, providing a robust and scalable solution for real-world crowd analysis.