ColoristaNet for Photorealistic Video Style Transfer
Xiaowen Qiu, Ruize Xu, Boan He, Yingtao Zhang, Wenqiang Zhang, Weifeng, Ge

TL;DR
ColoristaNet introduces a self-supervised, decoupled instance normalization approach combined with optical flow and ConvLSTM to enhance photorealistic video style transfer, achieving superior stylization and temporal coherence.
Contribution
It proposes a novel self-supervised framework with decoupled instance normalization and temporal coherence mechanisms for photorealistic video style transfer.
Findings
Outperforms state-of-the-art algorithms in stylization quality
Maintains photorealism effectively during style transfer
Ensures temporal coherence in video stylization
Abstract
Photorealistic style transfer aims to transfer the artistic style of an image onto an input image or video while keeping photorealism. In this paper, we think it's the summary statistics matching scheme in existing algorithms that leads to unrealistic stylization. To avoid employing the popular Gram loss, we propose a self-supervised style transfer framework, which contains a style removal part and a style restoration part. The style removal network removes the original image styles, and the style restoration network recovers image styles in a supervised manner. Meanwhile, to address the problems in current feature transformation methods, we propose decoupled instance normalization to decompose feature transformation into style whitening and restylization. It works quite well in ColoristaNet and can transfer image styles efficiently while keeping photorealism. To ensure temporal…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Image Enhancement Techniques
MethodsConvolution · Instance Normalization · Sigmoid Activation · Tanh Activation · ConvLSTM
