Learning to Navigate the Energy Landscape

TL;DR

This paper introduces a new architecture for computer vision that improves analysis by synthesis methods by providing multiple initial solutions and a navigational structure for efficient, gradient-free local search, achieving state-of-the-art results.

Contribution

It proposes a novel approach that enhances analysis by synthesis with multiple initial solutions and a navigational structure, surpassing traditional hybrid methods.

Findings

Achieved state-of-the-art results in RGB camera relocalization.

Demonstrated generalizability on hand pose estimation and image retrieval.

Introduced a new large-scale dataset for challenging relocalization tasks.

Abstract

In this paper, we present a novel and efficient architecture for addressing computer vision problems that use `Analysis by Synthesis'. Analysis by synthesis involves the minimization of the reconstruction error which is typically a non-convex function of the latent target variables. State-of-the-art methods adopt a hybrid scheme where discriminatively trained predictors like Random Forests or Convolutional Neural Networks are used to initialize local search algorithms. While these methods have been shown to produce promising results, they often get stuck in local optima. Our method goes beyond the conventional hybrid architecture by not only proposing multiple accurate initial solutions but by also defining a navigational structure over the solution space that can be used for extremely efficient gradient-free local search. We demonstrate the efficacy of our approach on the challenging problem of RGB Camera Relocalization. To make the RGB camera relocalization problem particularly challenging, we introduce a new dataset of 3D environments which are significantly larger than those found in other publicly-available datasets. Our experiments reveal that the proposed method is able to achieve state-of-the-art camera relocalization results. We also demonstrate the generalizability of our approach on Hand Pose Estimation and Image Retrieval tasks.