LucidDream: Controlled Temporally-Consistent DeepDream on Videos

TL;DR

This paper introduces techniques to enhance control and reduce flickering artifacts in DeepDream video applications, enabling more consistent and aesthetically pleasing hallucinations through temporal consistency and class control modifications.

Contribution

It presents a simple method to improve class control and introduces a temporal consistency loss to mitigate flickering in DeepDream videos.

Findings

Enhanced control over hallucinated object classes.

Reduced flickering artifacts in DeepDream videos.

Improved aesthetic appeal of video outputs.

Abstract

In this work, we aim to propose a set of techniques to improve the controllability and aesthetic appeal when DeepDream, which uses a pre-trained neural network to modify images by hallucinating objects into them, is applied to videos. In particular, we demonstrate a simple modification that improves control over the class of object that DeepDream is induced to hallucinate. We also show that the flickering artifacts which frequently appear when DeepDream is applied on videos can be mitigated by the use of an additional temporal consistency loss term.