Gaussian Process Priors for View-Aware Inference

TL;DR

This paper introduces a probabilistic framework using Gaussian process priors to incorporate view-dependent correlations in vision tasks, enhancing capabilities like novel view synthesis and latent space prediction.

Contribution

It develops a new view kernel for SO(3) and demonstrates how probabilistic priors can improve inference in view-aware computer vision applications.

Findings

The proposed view kernel generalizes the periodic kernel on SO(3).

Probabilistic priors improve novel view synthesis accuracy.

Framework enables prediction in generative model latent spaces.

Abstract

While frame-independent predictions with deep neural networks have become the prominent solutions to many computer vision tasks, the potential benefits of utilizing correlations between frames have received less attention. Even though probabilistic machine learning provides the ability to encode correlation as prior knowledge for inference, there is a tangible gap between the theory and practice of applying probabilistic methods to modern vision problems. For this, we derive a principled framework to combine information coupling between camera poses (translation and orientation) with deep models. We proposed a novel view kernel that generalizes the standard periodic kernel in $\mathrm{SO}(3)$. We show how this soft-prior knowledge can aid several pose-related vision tasks like novel view synthesis and predict arbitrary points in the latent space of generative models, pointing towards a range of new applications for inter-frame reasoning.