What Physics do Data-Driven MoCap-to-Radar Models Learn?

TL;DR

This paper evaluates whether data-driven MoCap-to-radar models truly learn the physics of Doppler effects, introducing interpretability metrics to assess physical consistency without needing radar data.

Contribution

It proposes a physics-based interpretability framework with two metrics to evaluate the physical realism of MoCap-to-radar models.

Findings

Low reconstruction error does not ensure physical accuracy.

Transformer models with temporal attention better learn underlying physics.

Models can perform well visually but fail physics-based tests.

Abstract

Data-driven MoCap-to-radar models generate plausible micro-Doppler spectrograms, but do they actually learn the underlying physics? We introduce a physics-based interpretability framework to answer this question via two proposed complementary metrics: one measures alignment between model predictions and the physics-derived Doppler frequency, while the other tests whether predictions preserve the velocity-frequency relationship under velocity intervention. Both metrics require only MoCap input and model predictions, without access to measured radar data. Experiments across several model architectures reveal that low reconstruction error does not guarantee physical consistency: some, but not all, models achieve low error yet perform poorly on the two physics-based metrics. Further analysis shows that temporal attention is critical for transformer-based models to learn the underlying physics.