Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias

TL;DR

This paper introduces techniques to improve monocular depth prediction by addressing camera pose distribution bias through pose-aware data augmentation and conditional modeling, enhancing accuracy on uncommon and unseen camera poses.

Contribution

The paper proposes two novel methods—pose-aware data augmentation and pose encoding—that mitigate pose bias and improve depth prediction generalization.

Findings

Enhanced depth prediction on uncommon camera poses

Improved generalization to unseen camera angles

Applicable across various predictor architectures

Abstract

Monocular depth predictors are typically trained on large-scale training sets which are naturally biased w.r.t the distribution of camera poses. As a result, trained predictors fail to make reliable depth predictions for testing examples captured under uncommon camera poses. To address this issue, we propose two novel techniques that exploit the camera pose during training and prediction. First, we introduce a simple perspective-aware data augmentation that synthesizes new training examples with more diverse views by perturbing the existing ones in a geometrically consistent manner. Second, we propose a conditional model that exploits the per-image camera pose as prior knowledge by encoding it as a part of the input. We show that jointly applying the two methods improves depth prediction on images captured under uncommon and even never-before-seen camera poses. We show that our methods improve performance when applied to a range of different predictor architectures. Lastly, we show that explicitly encoding the camera pose distribution improves the generalization performance of a synthetically trained depth predictor when evaluated on real images.