A Deep Learning Ensemble Framework for Off-Nadir Geocentric Pose Prediction

TL;DR

This paper introduces a deep learning ensemble framework that accurately predicts the geocentric pose of aerial vehicles from off-nadir satellite images, enhancing disaster response capabilities.

Contribution

It presents a novel ensemble approach combining U-Net and convolutional models to improve geocentric pose prediction from satellite imagery.

Findings

Achieved R2=0.917 in pose prediction accuracy

Outperformed previous methodologies significantly

Demonstrated the usefulness of elevation masks in feature engineering

Abstract

Computational methods to accelerate natural disaster response include change detection, map alignment, and vision-aided navigation. Current software functions optimally only on near-nadir images, though off-nadir images are often the first sources of information following a natural disaster. The use of off-nadir images for the aforementioned tasks requires the computation of geocentric pose, which is an aerial vehicle's spatial orientation with respect to gravity. This study proposes a deep learning ensemble framework to predict geocentric pose using 5,923 near-nadir and off-nadir RGB satellite images of cities worldwide. First, a U-Net Fully Convolutional Neural Network predicts the pixel-wise above-ground elevation mask of the RGB images. Then, the elevation masks are concatenated with the RGB images to form four-channel inputs fed into a second convolutional model, which predicts orientation angle and magnification scale. A performance accuracy of R2=0.917 significantly outperforms previous methodologies. In addition, outlier removal is performed through supervised interpolation, and a sensitivity analysis of elevation masks is conducted to gauge the usefulness of data features, motivating future avenues of feature engineering. The high-accuracy software built in this study contributes to mapping and navigation procedures for effective disaster response to save lives.