Fusion of Deep Neural Networks for Activity Recognition: A Regular Vine Copula Based Approach

TL;DR

This paper introduces a novel method for human activity recognition that combines multiple deep neural networks using regular vine copulas to model complex dependencies among sensor data, improving fusion accuracy.

Contribution

It presents a new fusion approach leveraging regular vine copulas to effectively integrate features from multiple neural networks in multi-sensor activity recognition.

Findings

Demonstrates improved recognition accuracy over baseline methods

Shows the effectiveness of vine copula-based feature fusion

Validates approach through numerical experiments

Abstract

In this paper, we propose regular vine copula based fusion of multiple deep neural network classifiers for the problem of multi-sensor based human activity recognition. We take the cross-modal dependence into account by employing regular vine copulas that are extremely flexible and powerful graphical models to characterize complex dependence among multiple modalities. Multiple deep neural networks are used to extract high-level features from multi-sensing modalities, with each deep neural network processing the data collected from a single sensor. The extracted high-level features are then combined using a regular vine copula model. Numerical experiments are conducted to demonstrate the effectiveness of our approach.