Palm up: Playing in the Latent Manifold for Unsupervised Pretraining

TL;DR

This paper introduces a novel unsupervised pretraining method that combines static dataset models with dynamic latent space exploration, enabling effective representation learning transferable to vision and reinforcement learning tasks.

Contribution

It proposes a new algorithm that integrates deep generative models with dynamic latent space exploration for unsupervised pretraining and reinforcement learning.

Findings

Learned representations transfer well to downstream tasks

The method enables exploration in static datasets using latent dynamics

Temporal pairing provides natural supervision for representation learning

Abstract

Large and diverse datasets have been the cornerstones of many impressive advancements in artificial intelligence. Intelligent creatures, however, learn by interacting with the environment, which changes the input sensory signals and the state of the environment. In this work, we aim to bring the best of both worlds and propose an algorithm that exhibits an exploratory behavior whilst it utilizes large diverse datasets. Our key idea is to leverage deep generative models that are pretrained on static datasets and introduce a dynamic model in the latent space. The transition dynamics simply mixes an action and a random sampled latent. It then applies an exponential moving average for temporal persistency, the resulting latent is decoded to image using pretrained generator. We then employ an unsupervised reinforcement learning algorithm to explore in this environment and perform unsupervised representation learning on the collected data. We further leverage the temporal information of this data to pair data points as a natural supervision for representation learning. Our experiments suggest that the learned representations can be successfully transferred to downstream tasks in both vision and reinforcement learning domains.