Multiple Style-Transfer in Real-Time

TL;DR

This paper extends real-time style transfer to incorporate multiple style images simultaneously, capturing diverse style features and improving temporal stability in video sequences.

Contribution

It introduces a novel framework for multi-style transfer in real-time and discusses techniques to enhance temporal consistency in stylized videos.

Findings

Successfully combines features from multiple style images.

Achieves color preservation from different styles.

Discusses techniques for improved temporal stability.

Abstract

Style transfer aims to combine the content of one image with the artistic style of another. It was discovered that lower levels of convolutional networks captured style information, while higher levels captures content information. The original style transfer formulation used a weighted combination of VGG-16 layer activations to achieve this goal. Later, this was accomplished in real-time using a feed-forward network to learn the optimal combination of style and content features from the respective images. The first aim of our project was to introduce a framework for capturing the style from several images at once. We propose a method that extends the original real-time style transfer formulation by combining the features of several style images. This method successfully captures color information from the separate style images. The other aim of our project was to improve the temporal style continuity from frame to frame. Accordingly, we have experimented with the temporal stability of the output images and discussed the various available techniques that could be employed as alternatives.