Vital Sign Monitoring in Dynamic Environment via mmWave Radar and Camera Fusion

TL;DR

This paper presents a fusion system combining mmWave radar and camera data to reliably monitor vital signs in dynamic environments, overcoming movement and environmental challenges with advanced signal processing.

Contribution

It introduces a novel multi-modal fusion approach with a specialized WMC-VMD algorithm for robust vital sign detection amidst movement and environmental changes.

Findings

Respiration rate error < 0.5 RPM

Heartbeat rate error < 6 BPM

Effective in dynamic scenarios with subject movement

Abstract

Contact-free vital sign monitoring, which uses wireless signals for recognizing human vital signs (i.e, breath and heartbeat), is an attractive solution to health and security. However, the subject's body movement and the change in actual environments can result in inaccurate frequency estimation of heartbeat and respiratory. In this paper, we propose a robust mmWave radar and camera fusion system for monitoring vital signs, which can perform consistently well in dynamic scenarios, e.g., when some people move around the subject to be tracked, or a subject waves his/her arms and marches on the spot. Three major processing modules are developed in the system, to enable robust sensing. Firstly, we utilize a camera to assist a mmWave radar to accurately localize the subjects of interest. Secondly, we exploit the calculated subject position to form transmitting and receiving beamformers, which can improve the reflected power from the targets and weaken the impact of dynamic interference. Thirdly, we propose a weighted multi-channel Variational Mode Decomposition (WMC-VMD) algorithm to separate the weak vital sign signals from the dynamic ones due to subject's body movement. Experimental results show that, the 90${^{th}}$ percentile errors in respiration rate (RR) and heartbeat rate (HR) are less than 0.5 RPM (respirations per minute) and 6 BPM (beats per minute), respectively.