DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognition

TL;DR

This paper introduces a new large-scale mmWave radar dataset for heterogeneous human action recognition and proposes DAP-Net, a novel Doppler-aware network that improves cross-source generalization and robustness.

Contribution

The paper presents UniMM-HAR, the first large-scale heterogeneous mmWave dataset, and proposes DAP-Net, a Doppler-aware network with novel modules for source-invariant action recognition.

Findings

DAP-Net outperforms existing methods in heterogeneous settings.

Achieves state-of-the-art accuracy on UniMM-HAR.

Demonstrates strong cross-source robustness.

Abstract

Millimeter-wave (mmWave) radar provides privacy-preserving sensing and is valuable for human action recognition (HAR). Existing mmWave point cloud datasets are limited in scale and mostly collected under homogeneous single-source settings, preventing current methods from handling real-world distribution shifts caused by heterogeneous radar sources, such as different devices and frequency bands. To address this, we introduce UniMM-HAR, the largest and first mmWave point cloud HAR dataset for heterogeneous multi-source scenarios, standardizing three distinct radar configurations to realistically evaluate cross-source generalization. We further propose the Doppler-aware Point Cloud Network (DAP-Net) to tackle heterogeneity challenges. DAP-Net enhances intra-modal representations and performs cross-modal alignment to learn source-invariant action semantics. Leveraging action-consistent spatio-temporal Doppler patterns as anchors, the Dual-space Doppler Reparameterization (D2R) module performs sample-adaptive geometric densification and Doppler-guided feature recalibration, while the Text Alignment Module (TAM) provides stable semantic anchors via a pretrained textual space. Experiments show that DAP-Net significantly outperforms existing methods under heterogeneous radar settings, achieving state-of-the-art accuracy and strong cross-source robustness.