Data Generation as Sequential Decision Making

TL;DR

This paper presents a unified view of generative models as sequential decision processes, introducing neural network-based policies for data imputation via an MDP framework and guided policy search.

Contribution

It extends existing models by framing data imputation as an MDP and develops neural policies trained with guided policy search for effective data completion.

Findings

Models successfully perform data imputation across various datasets.

The approach effectively handles varying difficulty levels in imputation tasks.

Iterative feedback improves prediction quality.

Abstract

We connect a broad class of generative models through their shared reliance on sequential decision making. Motivated by this view, we develop extensions to an existing model, and then explore the idea further in the context of data imputation -- perhaps the simplest setting in which to investigate the relation between unconditional and conditional generative modelling. We formulate data imputation as an MDP and develop models capable of representing effective policies for it. We construct the models using neural networks and train them using a form of guided policy search. Our models generate predictions through an iterative process of feedback and refinement. We show that this approach can learn effective policies for imputation problems of varying difficulty and across multiple datasets.