Real-time multiview data fusion for object tracking with RGBD sensors

TL;DR

This paper introduces a real-time multiview RGBD sensor fusion system for accurate vehicle tracking, addressing sensor wear and motion correction to enhance robustness and speed.

Contribution

A novel correction method for depth sensor shift and a sensor-wise filtering system for motion correction, enabling high-speed, accurate multiview object tracking.

Findings

Operates at up to 25 fps with five cameras

Achieves high accuracy and robustness in vehicle tracking

Effectively corrects sensor wear and motion uncertainties

Abstract

This paper presents a new approach to accurately track a moving vehicle with a multiview setup of red-green-blue depth (RGBD) cameras. We first propose a correction method to eliminate a shift, which occurs in depth sensors when they become worn. This issue could not be otherwise corrected with the ordinary calibration procedure. Next, we present a sensor-wise filtering system to correct for an unknown vehicle motion. A data fusion algorithm is then used to optimally merge the sensor-wise estimated trajectories. We implement most parts of our solution in the graphic processor. Hence, the whole system is able to operate at up to 25 frames per second with a configuration of five cameras. Test results show the accuracy we achieved and the robustness of our solution to overcome uncertainties in the measurements and the modelling.