Conditional COT-GAN for Video Prediction with Kernel Smoothing

TL;DR

This paper introduces a conditional COT-GAN model for sequence prediction, especially video forecasting, by incorporating kernel smoothing to improve convergence, advancing the use of causal optimal transport in sequential learning.

Contribution

It develops a conditional version of COT-GAN for sequence prediction and enhances convergence analysis using kernel smoothing techniques.

Findings

Successful application to video prediction tasks

Improved convergence properties demonstrated

Effective modeling of sequence evolution given past data

Abstract

Causal Optimal Transport (COT) results from imposing a temporal causality constraint on classic optimal transport problems, which naturally generates a new concept of distances between distributions on path spaces. The first application of the COT theory for sequential learning was given in Xu et al. (2020), where COT-GAN was introduced as an adversarial algorithm to train implicit generative models optimized for producing sequential data. Relying on (Xu et al., 2020), the contribution of the present paper is twofold. First, we develop a conditional version of COT-GAN suitable for sequence prediction. This means that the dataset is now used in order to learn how a sequence will evolve given the observation of its past evolution. Second, we improve on the convergence results by working with modifications of the empirical measures via kernel smoothing due to (Pflug and Pichler (2016)). The resulting kernel conditional COT-GAN algorithm is illustrated with an application for video prediction.