Neural Style Transfer for Computer Games

TL;DR

This paper introduces a depth-aware neural style transfer method integrated into 3D game rendering, achieving consistent stylisation and outperforming existing image and video NST techniques.

Contribution

It presents a novel approach for embedding neural style transfer directly into 3D game rendering pipelines, ensuring temporal consistency and higher quality stylisation.

Findings

Achieves temporally consistent stylised game scenes

Outperforms state-of-the-art image and video NST methods

Validates effectiveness through qualitative and quantitative experiments

Abstract

Neural Style Transfer (NST) research has been applied to images, videos, 3D meshes and radiance fields, but its application to 3D computer games remains relatively unexplored. Whilst image and video NST systems can be used as a post-processing effect for a computer game, this results in undesired artefacts and diminished post-processing effects. Here, we present an approach for injecting depth-aware NST as part of the 3D rendering pipeline. Qualitative and quantitative experiments are used to validate our in-game stylisation framework. We demonstrate temporally consistent results of artistically stylised game scenes, outperforming state-of-the-art image and video NST methods.