An Order Preserving Bilinear Model for Person Detection in Multi-Modal Data

TL;DR

This paper introduces an order preserving bilinear model that effectively combines low-resolution video and seismic data for person detection, maintaining spatial-temporal order to improve accuracy in multi-modal settings.

Contribution

The paper presents a novel order preserving bilinear framework that enhances multi-modal person detection by leveraging low-resolution video and seismic data, outperforming existing fusion methods.

Findings

Outperforms recent orderless bilinear models.

Effectively utilizes low-resolution video for detection.

Maintains spatio-temporal order in feature fusion.

Abstract

We propose a new order preserving bilinear framework that exploits low-resolution video for person detection in a multi-modal setting using deep neural networks. In this setting cameras are strategically placed such that less robust sensors, e.g. geophones that monitor seismic activity, are located within the field of views (FOVs) of cameras. The primary challenge is being able to leverage sufficient information from videos where there are less than 40 pixels on targets, while also taking advantage of less discriminative information from other modalities, e.g. seismic. Unlike state-of-the-art methods, our bilinear framework retains spatio-temporal order when computing the vector outer products between pairs of features. Despite the high dimensionality of these outer products, we demonstrate that our order preserving bilinear framework yields better performance than recent orderless bilinear models and alternative fusion methods.