ScaleNet: A Shallow Architecture for Scale Estimation

TL;DR

ScaleNet is a novel shallow neural network architecture that accurately estimates scale factors between images using dilated convolutions and correlation layers, improving various computer vision tasks like pose estimation and 3D reconstruction.

Contribution

The paper introduces ScaleNet, a new architecture for scale estimation that leverages dilated convolutions and correlation layers, enhancing performance across multiple vision benchmarks.

Findings

Significant performance improvements in camera pose estimation and 3D reconstruction.

Effective combination of ScaleNet with local features and dense correspondence networks.

Comprehensive evaluation demonstrating the efficiency and accuracy of ScaleNet.

Abstract

In this paper, we address the problem of estimating scale factors between images. We formulate the scale estimation problem as a prediction of a probability distribution over scale factors. We design a new architecture, ScaleNet, that exploits dilated convolutions as well as self and cross-correlation layers to predict the scale between images. We demonstrate that rectifying images with estimated scales leads to significant performance improvements for various tasks and methods. Specifically, we show how ScaleNet can be combined with sparse local features and dense correspondence networks to improve camera pose estimation, 3D reconstruction, or dense geometric matching in different benchmarks and datasets. We provide an extensive evaluation on several tasks and analyze the computational overhead of ScaleNet. The code, evaluation protocols, and trained models are publicly available at https://github.com/axelBarroso/ScaleNet.