Adaptive Sampling-based Particle Filter for Visual-inertial Gimbal in the Wild

TL;DR

This paper introduces an adaptive sampling particle filter for a drone gimbal that uses vision-based cues like sky and ground to achieve robust camera stabilization in natural environments.

Contribution

It presents a novel geometry-based visual tracking method combined with a lightweight neural network and adaptive particle sampling for improved drone gimbal stabilization.

Findings

Successful real-world testing on a drone gimbal in natural landscapes.

Robust camera stabilization using skyline and ground plane references.

Efficient segmentation with ResNet-18 on Jetson Nano.

Abstract

In this paper, we present a Computer Vision (CV) based tracking and fusion algorithm, dedicated to a 3D printed gimbal system on drones operating in nature. The whole gimbal system can stabilize the camera orientation robustly in a challenging nature scenario by using skyline and ground plane as references. Our main contributions are the following: a) a light-weight Resnet-18 backbone network model was trained from scratch, and deployed onto the Jetson Nano platform to segment the image into binary parts (ground and sky); b) our geometry assumption from nature cues delivers the potential for robust visual tracking by using the skyline and ground plane as a reference; c) a spherical surface-based adaptive particle sampling, can fuse orientation from multiple sensor sources flexibly. The whole algorithm pipeline is tested on our customized gimbal module including Jetson and other hardware components. The experiments were performed on top of a building in the real landscape.