MoCap2Radar: A Spatiotemporal Transformer for Synthesizing Micro-Doppler Radar Signatures from Motion Capture

TL;DR

This paper introduces MoCap2Radar, a transformer-based model that synthesizes radar spectrograms from MoCap data, enabling realistic radar signal generation with less computation and potential for data augmentation.

Contribution

The paper presents a novel transformer-based approach for translating MoCap data into radar spectrograms, demonstrating improved realism and generalizability over physics-based methods.

Findings

Produces plausible radar spectrograms from MoCap data

Achieves good generalization in real-world experiments

Requires less computation than physics-based methods

Abstract

We present a pure machine learning process for synthesizing radar spectrograms from Motion-Capture (MoCap) data. We formulate MoCap-to-spectrogram translation as a windowed sequence-to-sequence task using a transformer-based model that jointly captures spatial relations among MoCap markers and temporal dynamics across frames. Real-world experiments show that the proposed approach produces visually and quantitatively plausible doppler radar spectrograms and achieves good generalizability. Ablation experiments show that the learned model includes both the ability to convert multi-part motion into doppler signatures and an understanding of the spatial relations between different parts of the human body. The result is an interesting example of using transformers for time-series signal processing. It is especially applicable to edge computing and Internet of Things (IoT) radars. It also suggests the ability to augment scarce radar datasets using more abundant MoCap data for training higher-level applications. Finally, it requires far less computation than physics-based methods for generating radar data.