On-Board Visual Tracking with Unmanned Aircraft System (UAS)

TL;DR

This paper introduces a real-time visual tracking system for small UAVs that uses normalized cross correlation and Kalman filtering, capable of autonomously tracking objects in diverse conditions with re-detection features.

Contribution

The paper presents a novel onboard UAV tracking system that combines zero mean normalized cross correlation with Kalman filtering for efficient real-time object tracking.

Findings

System operates in real-time on a 1.2 GHz PC/104 onboard a UAV.

Successfully tracks diverse objects both in lab and field tests.

Capable of re-detecting objects after tracking failure.

Abstract

This paper presents the development of a real time tracking algorithm that runs on a 1.2 GHz PC/104 computer on-board a small UAV. The algorithm uses zero mean normalized cross correlation to detect and locate an object in the image. A kalman filter is used to make the tracking algorithm computationally efficient. Object position in an image frame is predicted using the motion model and a search window, centered at the predicted position is generated. Object position is updated with the measurement from object detection. The detected position is sent to the motion controller to move the gimbal so that the object stays at the center of the image frame. Detection and tracking is autonomously carried out on the payload computer and the system is able to work in two different methods. The first method starts detecting and tracking using a stored image patch. The second method allows the operator on the ground to select the interest object for the UAV to track. The system is capable of re-detecting an object, in the event of tracking failure. Performance of the tracking system was verified both in the lab and on the field by mounting the payload on a vehicle and simulating a flight. Tests show that the system can detect and track a diverse set of objects in real time. Flight testing of the system will be conducted at the next available opportunity.