Reducing Drift in Visual Odometry by Inferring Sun Direction Using a Bayesian Convolutional Neural Network

TL;DR

This paper introduces a Bayesian CNN-based method to infer sun direction from images for improving visual odometry accuracy, even when the sun isn't visible, by incorporating uncertainty estimates into the pipeline.

Contribution

It presents a novel Bayesian CNN model for sun detection that provides uncertainty estimates, enhancing visual odometry performance without requiring visible sun cues.

Findings

Median sun direction error of 12 degrees on KITTI dataset

Up to 42% improvement in translational accuracy

Up to 32% improvement in rotational accuracy

Abstract

We present a method to incorporate global orientation information from the sun into a visual odometry pipeline using only the existing image stream, where the sun is typically not visible. We leverage recent advances in Bayesian Convolutional Neural Networks to train and implement a sun detection model that infers a three-dimensional sun direction vector from a single RGB image. Crucially, our method also computes a principled uncertainty associated with each prediction, using a Monte Carlo dropout scheme. We incorporate this uncertainty into a sliding window stereo visual odometry pipeline where accurate uncertainty estimates are critical for optimal data fusion. Our Bayesian sun detection model achieves a median error of approximately 12 degrees on the KITTI odometry benchmark training set, and yields improvements of up to 42% in translational ARMSE and 32% in rotational ARMSE compared to standard VO. An open source implementation of our Bayesian CNN sun estimator (Sun-BCNN) using Caffe is available at https://github. com/utiasSTARS/sun-bcnn-vo