Joint Estimation of Image Representations and their Lie Invariants

TL;DR

This paper introduces two theoretical methods for jointly estimating image representations and their Lie group-based dynamics, enabling interpolation and extrapolation of image sequences while disentangling content from state information.

Contribution

It presents novel probabilistic PCA and nonlinear PCA approaches that incorporate Lie group dynamics to improve image sequence analysis and representation disentanglement.

Findings

Probabilistic PCA provides a linear subspace model for image representations.

Probabilistic nonlinear PCA relaxes linear assumptions with variational methods.

Explicit modeling of Lie group dynamics enables smooth and composable transformations.

Abstract

Images encode both the state of the world and its content. The former is useful for tasks such as planning and control, and the latter for classification. The automatic extraction of this information is challenging because of the high-dimensionality and entangled encoding inherent to the image representation. This article introduces two theoretical approaches aimed at the resolution of these challenges. The approaches allow for the interpolation and extrapolation of images from an image sequence by joint estimation of the image representation and the generators of the sequence dynamics. In the first approach, the image representations are learned using probabilistic PCA \cite{tipping1999probabilistic}. The linear-Gaussian conditional distributions allow for a closed form analytical description of the latent distributions but assumes the underlying image manifold is a linear subspace. In the second approach, the image representations are learned using probabilistic nonlinear PCA which relieves the linear manifold assumption at the cost of requiring a variational approximation of the latent distributions. In both approaches, the underlying dynamics of the image sequence are modelled explicitly to disentangle them from the image representations. The dynamics themselves are modelled with Lie group structure which enforces the desirable properties of smoothness and composability of inter-image transformations.