Interactive Control over Temporal Consistency while Stylizing Video Streams
Sumit Shekhar, Max Reimann, Moritz Hilscher, Amir Semmo, J\"urgen, D\"ollner, Matthias Trapp

TL;DR
This paper introduces a real-time, interactive video stylization method that maintains artistic aesthetic while controlling temporal consistency, using a lightweight optical-flow network and adaptive feature combination.
Contribution
It presents a novel real-time video stylization approach with interactive control over temporal consistency, supporting full HD resolution and online processing.
Findings
Operates at 80 FPS on desktop systems
Outperforms state-of-the-art methods in consistency quality
Supports interactive selection of consistency features
Abstract
Image stylization has seen significant advancement and widespread interest over the years, leading to the development of a multitude of techniques. Extending these stylization techniques, such as Neural Style Transfer (NST), to videos is often achieved by applying them on a per-frame basis. However, per-frame stylization usually lacks temporal consistency, expressed by undesirable flickering artifacts. Most of the existing approaches for enforcing temporal consistency suffer from one or more of the following drawbacks: They (1) are only suitable for a limited range of techniques, (2) do not support online processing as they require the complete video as input, (3) cannot provide consistency for the task of stylization, or (4) do not provide interactive consistency control. Domain-agnostic techniques for temporal consistency aim to eradicate flickering completely but typically disregard…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Computer Graphics and Visualization Techniques · Advanced Vision and Imaging
