# Improving Image-Based Localization with Deep Learning: The Impact of the Loss Function

## TL;DR

This paper explores how the choice of loss function affects neural network performance in monocular image localization, proposing a new combined loss term that improves accuracy by considering the coupled nature of position and rotation errors.

## Contribution

It introduces a novel loss term that jointly optimizes position and rotation errors, enhancing pose regression accuracy in indoor scene localization.

## Key findings

- Up to 26.7% reduction in median positional error
- Up to 24.0% reduction in median rotational error
- Improved localization accuracy over baseline PoseNet

## Abstract

This work investigates the impact of the loss function on the performance of Neural Networks, in the context of a monocular, RGB-only, image localization task. A common technique used when regressing a camera's pose from an image is to formulate the loss as a linear combination of positional and rotational mean squared error (using tuned hyperparameters as coefficients). In this work we observe that changes to rotation and position mutually affect the captured image, and in order to improve performance, a pose regression network's loss function should include a term which combines the error of both of these coupled quantities. Based on task specific observations and experimental tuning, we present said loss term, and create a new model by appending this loss term to the loss function of the pre-existing pose regression network `PoseNet'. We achieve improvements in the localization accuracy of the network for indoor scenes; with decreases of up to 26.7% and 24.0% in the median positional and rotational error respectively, when compared to the default PoseNet.