Broadband tunable microwave photonic radar for simultaneous detection of human respiration, heartbeat, and speech with deep learning-based speech recognition

TL;DR

This paper introduces a broadband tunable microwave photonic radar system capable of simultaneously monitoring human respiration, heartbeat, and speech, with integrated deep learning for high-accuracy speech recognition across multiple frequency bands.

Contribution

It presents a novel broadband, frequency-tunable radar system that combines physiological monitoring with deep learning-based speech recognition, enhancing environmental adaptability and multi-modal sensing capabilities.

Findings

Achieved over 97% speech recognition accuracy in Ku, K, and Ka bands.

Demonstrated reliable detection of respiration and heartbeat with minimal error.

Validated system performance through experiments with simulated vital signs.

Abstract

Multimodal vital sign monitoring and speech detection hold significant importance in medical health, public safety, and other fields. This study proposes a broadband tunable microwave photonic radar system that can simultaneously monitor respiration, heartbeat, and speech. The system works by generating broadband radar signals to detect subtle skin displacements caused by these physiological activities. It then utilizes phase variations in radar echo signals to extract and reconstruct the corresponding physiological signals. In order to enhance the processing capability for speech signals, a convolutional neural network with a dual-channel feature fusion model is incorporated, enabling high-precision speech recognition. In addition, the system's frequency-tunable characteristic allows it to flexibly switch frequency bands to adapt to different working environments, greatly improving its practicality and environmental adaptability. In concept-verification experiments, speech signals were reconstructed and recognized in the Ku, K, and Ka bands, achieving recognition accuracies of 97.20%, 98.07%, and 97.43%, respectively. The system's capability to detect multimodal vital signs was also thoroughly validated using a respiratory and heartbeat simulator. During a 20-second monitoring period, while accurately reconstructing speech, the maximum average error counts for respiratory and heartbeat monitoring were 0.39 and 0.87, respectively, proving its reliability and effectiveness in multimodal vital sign monitoring.