Long-Range Vision-Based UAV-assisted Localization for Unmanned Surface Vehicles

TL;DR

This paper introduces a UAV-assisted visual localization method for USVs in GPS-restricted marine environments, using deep learning, triangulation, and EKF to improve navigation accuracy during challenging conditions.

Contribution

It presents a novel UAV-based visual localization approach combining deep learning, geometric triangulation, and EKF, validated through real-world experiments in marine environments.

Findings

Effective USV localization without GPS in marine environments

Enhanced accuracy through camera angle adjustment and EKF integration

Successful real-world validation during MBZIRC-2024 challenge

Abstract

The global positioning system (GPS) has become an indispensable navigation method for field operations with unmanned surface vehicles (USVs) in marine environments. However, GPS may not always be available outdoors because it is vulnerable to natural interference and malicious jamming attacks. Thus, an alternative navigation system is required when the use of GPS is restricted or prohibited. To this end, we present a novel method that utilizes an Unmanned Aerial Vehicle (UAV) to assist in localizing USVs in GNSS-restricted marine environments. In our approach, the UAV flies along the shoreline at a consistent altitude, continuously tracking and detecting the USV using a deep learning-based approach on camera images. Subsequently, triangulation techniques are applied to estimate the USV's position relative to the UAV, utilizing geometric information and datalink range from the UAV. We propose adjusting the UAV's camera angle based on the pixel error between the USV and the image center throughout the localization process to enhance accuracy. Additionally, visual measurements are integrated into an Extended Kalman Filter (EKF) for robust state estimation. To validate our proposed method, we utilize a USV equipped with onboard sensors and a UAV equipped with a camera. A heterogeneous robotic interface is established to facilitate communication between the USV and UAV. We demonstrate the efficacy of our approach through a series of experiments conducted during the ``Muhammad Bin Zayed International Robotic Challenge (MBZIRC-2024)'' in real marine environments, incorporating noisy measurements and ocean disturbances. The successful outcomes indicate the potential of our method to complement GPS for USV navigation.