# Study on Multimodal Sensor Fusion for Heart Rate Estimation Using BCG and PPG Signals

## TL;DR

This study introduces a new method for estimating heart rate using BCG and PPG signals, offering a more comfortable and accurate alternative to traditional ECG for home monitoring.

## Contribution

The novel MM-TFNet model uses multimodal fusion of BCG and PPG signals with cross-modal attention for improved heart rate estimation.

## Key findings

- The model achieved a mean absolute error of 0.88 BPM in heart rate prediction.
- MM-TFNet outperformed existing deep learning methods in dynamic activity states.
- The approach shows potential for contactless, low-power home health monitoring systems.

## Abstract

Continuous heart rate monitoring is crucial for early cardiovascular disease detection. To overcome the discomfort and limitations of ECG in home settings, we propose a multimodal temporal fusion network (MM-TFNet) that integrates ballistocardiography (BCG) and photoplethysmography (PPG) signals. The network extracts temporal features from BCG and PPG signals through temporal convolutional networks (TCNs) and bidirectional long short-term memory networks (BiLSTMs), respectively, achieving cross-modal dynamic fusion at the feature level. First, bimodal features are projected into a unified dimensional space through fully connected layers. Subsequently, a cross-modal attention weight matrix is constructed for adaptive learning of the complementary correlation between BCG mechanical vibration and PPG volumetric flow features. Combined with dynamic focusing on key heartbeat waveforms through multi-head self-attention (MHSA), the model’s robustness under dynamic activity states is significantly enhanced. Experimental validation using a publicly available BCG-PPG-ECG simultaneous acquisition dataset comprising 40 subjects demonstrates that the model achieves excellent performance with a mean absolute error (MAE) of 0.88 BPM in heart rate prediction tasks, outperforming current mainstream deep learning methods. This study provides theoretical foundations and engineering guidance for developing contactless, low-power, edge-deployable home health monitoring systems, demonstrating the broad application potential of multimodal fusion methods in complex physiological signal analysis.