Unsupervised Transformation Learning via Convex Relaxations

TL;DR

This paper introduces an unsupervised method for learning meaningful image transformations, such as style or lighting changes, by reconstructing images from linear combinations of transformed neighbors, enabling high-quality modifications and extrapolation to new images.

Contribution

It presents a novel semiparametric approach that learns transformations without modeling data distribution, allowing for extrapolation and application to diverse images.

Findings

Produces high-quality transformed images on digits and portraits

Transforms extrapolate beyond training data

Method is unsupervised and does not require labeled data

Abstract

Our goal is to extract meaningful transformations from raw images, such as varying the thickness of lines in handwriting or the lighting in a portrait. We propose an unsupervised approach to learn such transformations by attempting to reconstruct an image from a linear combination of transformations of its nearest neighbors. On handwritten digits and celebrity portraits, we show that even with linear transformations, our method generates visually high-quality modified images. Moreover, since our method is semiparametric and does not model the data distribution, the learned transformations extrapolate off the training data and can be applied to new types of images.