Tracking in Aerial Hyperspectral Videos using Deep Kernelized Correlation Filters

TL;DR

This paper introduces DeepHKCF, a novel deep kernelized correlation filter tracker for aerial hyperspectral vehicle tracking, leveraging multi-modal sensors and synthetic data for improved accuracy and efficiency.

Contribution

The paper presents DeepHKCF, combining deep CNN features with kernelized correlation filters for aerial hyperspectral tracking, and introduces a synthetic dataset for vehicle classification in WAMI.

Findings

DeepHKCF outperforms existing trackers on synthetic hyperspectral videos.

The ROI mapping strategy significantly speeds up deep feature extraction.

A large synthetic dataset for vehicle classification is created and released.

Abstract

Hyperspectral imaging holds enormous potential to improve the state-of-the-art in aerial vehicle tracking with low spatial and temporal resolutions. Recently, adaptive multi-modal hyperspectral sensors have attracted growing interest due to their ability to record extended data quickly from aerial platforms. In this study, we apply popular concepts from traditional object tracking, namely (1) Kernelized Correlation Filters (KCF) and (2) Deep Convolutional Neural Network (CNN) features to aerial tracking in hyperspectral domain. We propose the Deep Hyperspectral Kernelized Correlation Filter based tracker (DeepHKCF) to efficiently track aerial vehicles using an adaptive multi-modal hyperspectral sensor. We address low temporal resolution by designing a single KCF-in-multiple Regions-of-Interest (ROIs) approach to cover a reasonably large area. To increase the speed of deep convolutional features extraction from multiple ROIs, we design an effective ROI mapping strategy. The proposed tracker also provides flexibility to couple with the more advanced correlation filter trackers. The DeepHKCF tracker performs exceptionally well with deep features set up in a synthetic hyperspectral video generated by the Digital Imaging and Remote Sensing Image Generation (DIRSIG) software. Additionally, we generate a large, synthetic, single-channel dataset using DIRSIG to perform vehicle classification in the Wide Area Motion Imagery (WAMI) platform. This way, the high-fidelity of the DIRSIG software is proved and a large scale aerial vehicle classification dataset is released to support studies on vehicle detection and tracking in the WAMI platform.