Adversarial Training for Adverse Conditions: Robust Metric Localisation using Appearance Transfer

TL;DR

This paper introduces an invertible image transformation method that enhances visual place recognition and metric localisation across diverse and adverse environmental conditions by generating condition-optimized images for feature matching.

Contribution

It proposes a novel invertible generator trained to produce synthetic images tailored for improved feature matching in localisation tasks under varying conditions.

Findings

Significantly improves localisation accuracy under changing conditions.

Reduces the number of mapping runs needed for reliable localisation.

Demonstrates robustness across a year-long dataset with diverse environmental changes.

Abstract

We present a method of improving visual place recognition and metric localisation under very strong appear- ance change. We learn an invertable generator that can trans- form the conditions of images, e.g. from day to night, summer to winter etc. This image transforming filter is explicitly designed to aid and abet feature-matching using a new loss based on SURF detector and dense descriptor maps. A network is trained to output synthetic images optimised for feature matching given only an input RGB image, and these generated images are used to localize the robot against a previously built map using traditional sparse matching approaches. We benchmark our results using multiple traversals of the Oxford RobotCar Dataset over a year-long period, using one traversal as a map and the other to localise. We show that this method significantly improves place recognition and localisation under changing and adverse conditions, while reducing the number of mapping runs needed to successfully achieve reliable localisation.